Scene & Task Parsing

Scene & Task Parsing — Natural Language → Training Setup

MH-FLOCKE training starts with a natural language scene description like “dog plays with ball on grass.” The system parses this into a structured task with environment, terrain, and behavior configuration.

Pipeline

1. TaskParser (llm_bridge.py) — Extracts task type, environment, difficulty, objects from the scene description. Uses pattern matching with optional LLM fallback.
2. UnderstandEngine (understand.py) — Generates behavior knowledge appropriate for the scene. Maps scene objects to interaction patterns.
3. SceneInstruction (scene_instruction.py) — Predefined scene templates with terrain type, ball parameters, curriculum stages. E.g. “walk on flat meadow” → flat terrain, no ball, locomotion reward only.
4. BehaviorKnowledge — Scene-specific behaviors loaded: ball scene adds “chase,” “sniff,” “play.” Hill scene adds “balance_slope,” “cautious_walk.”

Scene Templates

SCENE_INSTRUCTIONS = {
  "walk on flat meadow": {terrain: "flat", ball: false},
  "dog plays with ball": {terrain: "flat", ball: true, curriculum: 5-stage},
  "walk on hilly terrain": {terrain: "hills", difficulty: 0.5},
  ...more
}

API Reference

TaskParser

parse(text: str) → TaskDescription

Returns task_type, environment, difficulty, objects, terrain_type.

UnderstandEngine

understand_task(description, creature_type) → dict

Knowledge acquisition: returns behaviors, terrain config, reward components.

SceneInstruction

SCENE_INSTRUCTIONS: dict  — predefined scene templates
get_instruction(scene_text) → dict  — match or generate