RLHF-Blender: A Configurable Interactive Interface for Learning from Diverse Human Feedback
METADATA ONLY
Loading...
Author / Producer
Date
2023
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
To use reinforcement learning from human feedback (RLHF) in practical applications, it is crucial to learn reward models from diverse sources of human feedback and to consider human factors involved in providing feedback of different types. However, the systematic study of learning from diverse types of feedback is held back by limited standardized tooling available to researchers. To bridge this gap, we propose RLHF-Blender, a configurable, interactive interface for learning from human feedback. RLHF-Blender provides a modular experimentation framework and implementation that enables researchers to systematically investigate the properties and qualities of human feedback for reward learning. The system facilitates the exploration of various feedback types, including demonstrations, rankings, comparisons, and natural language instructions, as well as studies considering the impact of human factors on their effectiveness. We discuss a set of concrete research opportunities enabled by RLHF-Blender. More information is available at https://rlhfblender.info/.
Permanent link
Publication status
published
Editor
Book title
Interactive Learning with Implicit Human Feedback Workshop at ICML 2023
Journal / series
Volume
Pages / Article No.
Publisher
OpenReview
Event
Interactive Learning from Implicit Human Feedback Workshop @ ICML 2023
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Reinforcement learning; Human feedback; Human-AI communication; Human-in-the-loop learning
Organisational unit
09822 - El-Assady, Mennatallah / El-Assady, Mennatallah
Notes
Poster presented on July 29, 2023.
Funding
Related publications and datasets
Is new version of: 10.48550/ARXIV.2308.04332