- Description:
(Image, context, instruction) triplets manually taken and written by humans while ensuring that the instruction desirability can only be determined by looking at the image (validation set)
Homepage: https://asimov-benchmark.github.io/
Source code:
tfds.robotics.asimov.AsimovMultimodalManualVal
Versions:
0.1.0
(default): Initial release.
Download size:
Unknown size
Dataset size:
Unknown size
Auto-cached (documentation): Unknown
Splits:
Split | Examples |
---|
- Feature structure:
FeaturesDict({
'image': Image(shape=(None, None, 3), dtype=uint8),
'instructions': Sequence({
'context': Text(shape=(), dtype=string),
'instruction': Text(shape=(), dtype=string),
'prompt_with_constitution': Text(shape=(), dtype=string),
'prompt_with_constitution_chain_of_thought': Text(shape=(), dtype=string),
'prompt_with_constitution_chain_of_thought_antijailbreak': Text(shape=(), dtype=string),
'prompt_with_constitution_chain_of_thought_antijailbreak_adversary': Text(shape=(), dtype=string),
'prompt_with_constitution_chain_of_thought_antijailbreak_adversary_parts': Sequence(Text(shape=(), dtype=string)),
'prompt_with_constitution_chain_of_thought_antijailbreak_parts': Sequence(Text(shape=(), dtype=string)),
'prompt_with_constitution_chain_of_thought_parts': Sequence(Text(shape=(), dtype=string)),
'prompt_with_constitution_parts': Sequence(Text(shape=(), dtype=string)),
'prompt_without_constitution': Text(shape=(), dtype=string),
'prompt_without_constitution_parts': Sequence(Text(shape=(), dtype=string)),
'undesirable_groundtruth_answer': bool,
}),
'original_image': Image(shape=(None, None, 3), dtype=uint8),
})
- Feature documentation:
Feature | Class | Shape | Dtype | Description |
---|---|---|---|---|
FeaturesDict | ||||
image | Image | (None, None, 3) | uint8 | |
instructions | Sequence | |||
instructions/context | Text | string | ||
instructions/instruction | Text | string | ||
instructions/prompt_with_constitution | Text | string | ||
instructions/prompt_with_constitution_chain_of_thought | Text | string | ||
instructions/prompt_with_constitution_chain_of_thought_antijailbreak | Text | string | ||
instructions/prompt_with_constitution_chain_of_thought_antijailbreak_adversary | Text | string | ||
instructions/prompt_with_constitution_chain_of_thought_antijailbreak_adversary_parts | Sequence(Text) | (None,) | string | |
instructions/prompt_with_constitution_chain_of_thought_antijailbreak_parts | Sequence(Text) | (None,) | string | |
instructions/prompt_with_constitution_chain_of_thought_parts | Sequence(Text) | (None,) | string | |
instructions/prompt_with_constitution_parts | Sequence(Text) | (None,) | string | |
instructions/prompt_without_constitution | Text | string | ||
instructions/prompt_without_constitution_parts | Sequence(Text) | (None,) | string | |
instructions/undesirable_groundtruth_answer | Tensor | bool | ||
original_image | Image | (None, None, 3) | uint8 |
Supervised keys (See
as_supervised
doc):None
Figure (tfds.show_examples): Not supported.
Examples (tfds.as_dataframe): Missing.
Citation:
@article{sermanet2025asimov,
author = {Pierre Sermanet and Anirudha Majumdar and Alex Irpan and Dmitry Kalashnikov and Vikas Sindhwani},
title = {Generating Robot Constitutions & Benchmarks for Semantic Safety},
journal = {arXiv preprint arXiv:2503.08663},
url = {https://arxiv.org/abs/2503.08663},
year = {2025},
}