Dora Voice Control Node

Dora node that processes Spanish voice commands and translates them into robot actions. Supports multiple robot types via robot subfolders.

Features

Spanish voice command parsing (rule-based or LLM)
Robot adapter pattern for different gripper types
Real-time web debug interface
Command queue management
Workspace bounds validation

File Structure

dora_voice_control/
├── main.py                 # Thin orchestrator
│
├── core/                   # Shared logic
│   ├── behavior.py         # RobotBehavior with actions
│   ├── config.py           # Configuration classes
│   ├── node.py             # Dora adapter + dispatcher + context
│   ├── robot.py            # RobotAdapter base
│   ├── robot_io.py         # Pose/status/image handlers + command queue
│   ├── scene.py            # Scene state + notifier + objects handler
│   ├── state.py            # Thread-safe shared state
│   └── voice.py            # Voice input + parsing + intents
│
├── robots/                 # Robot-specific implementations
│   └── littlehand/         # Vacuum gripper robot
│       ├── adapter.py      # Vacuum adapter
│       ├── actions.py      # Action vocabulary
│       └── behavior.py     # Behavior binding
│
└── web/                    # Web interface
    ├── api.py              # FastAPI server
    ├── models.py           # Pydantic models
    └── templates.py        # HTML template

Robot Adapters

Set ROBOT_TYPE to select the robot package:

Type	Grab Command	Release Command
`littlehand` (alias: `vacuum`)	`vacuum_on`	`vacuum_off`

To add a new robot, create a new subfolder under robots/ with its adapter and behavior, then register it in robots/__init__.py.

Inputs/Outputs

Input	Type	Description
`voice_in`	string	Voice command text
`tcp_pose`	array	Robot pose [x, y, z, roll, pitch, yaw]
`objects`	JSON	Detected objects
`status`	JSON	Command execution status
`image_annotated`	array	Camera image

Output	Type	Description
`robot_cmd`	JSON	Robot command
`voice_out`	JSON	Response to user
`scene_update`	JSON	Scene state

Supported Commands (Spanish)

Command	Action	Example
`subir`	Move up	"sube"
`bajar`	Move down	"baja"
`tomar`	Grab object	"agarra el cubo rojo"
`soltar`	Release object	"suelta en la caja azul"
`ir`	Go to object	"ve al cilindro"
`reiniciar`	Reset	"reinicia"

Environment Variables

# Robot Configuration
ROBOT_TYPE=littlehand       # "littlehand" (alias: "vacuum")

# Web API
API_ENABLED=true
API_PORT=9001

# TCP Parameters
TCP_OFFSET_MM=63.0
APPROACH_OFFSET_MM=50.0
STEP_MM=20.0

# LLM (optional)
LLM_PROVIDER=rules          # "rules", "gemini", "ollama"
GOOGLE_API_KEY=your_key

# Initial Position
INIT_ON_START=true
INIT_X=300.0
INIT_Y=0.0
INIT_Z=350.0

# Safety
DRY_RUN=false
WORKSPACE_MIN_Z=0
WORKSPACE_MAX_Z=500

Web Debug Interface

Access at http://localhost:8080:

Camera view with detections
Real-time status (pose, objects, queue)
Send manual commands
View parse results

API Endpoints

# Status
curl http://localhost:8080/api/status

# Objects
curl http://localhost:8080/api/objects

# Send command
curl -X POST http://localhost:8080/api/command \
  -H "Content-Type: application/json" \
  -d '{"text": "sube"}'

# Clear queue
curl -X POST http://localhost:8080/api/queue/clear

Dataflow Example

- id: voice
  build: uv pip install -e dora_voice_control
  path: dora_voice_control/dora_voice_control/main.py
  env:
    ROBOT_TYPE: "vacuum"
    API_ENABLED: "true"
  inputs:
    voice_in: iobridge/text_out
    tcp_pose: robot/tcp_pose
    objects: detector/objects
    status: robot/status
  outputs:
    - robot_cmd
    - voice_out
    - scene_update

Adding a New Robot

Create dora_voice_control/dora_voice_control/robots/<robot_name>/ with:

adapter.py implementing a RobotAdapter
actions.py defining action aliases (can reuse defaults)
behavior.py binding the behavior class

4.4 KiB Raw Permalink Blame History