179 lines
3.4 KiB
Markdown
179 lines
3.4 KiB
Markdown
# Dora IOBridge Node
|
|
|
|
A WebSocket server that bridges web clients with the Dora dataflow for real-time voice commands and scene updates.
|
|
|
|
## Inputs/Outputs
|
|
|
|
| Input | Type | Description |
|
|
|----------------|--------|---------------------------------------|
|
|
| `voice_out` | JSON | Response from voice control node |
|
|
| `scene_update` | JSON | Scene objects from voice control |
|
|
|
|
| Output | Type | Description |
|
|
|----------------|--------|---------------------------------------|
|
|
| `voice_in` | string | Voice commands forwarded to Dora |
|
|
|
|
## Environment Variables
|
|
|
|
```bash
|
|
VOICE_HOST=0.0.0.0 # Bind address
|
|
VOICE_PORT=8765 # Listen port
|
|
```
|
|
|
|
## Installation
|
|
|
|
```bash
|
|
cd dora_iobridge
|
|
pip install -e .
|
|
```
|
|
|
|
## Testing
|
|
|
|
### Test with WebSocket (wscat)
|
|
|
|
```bash
|
|
# Install wscat
|
|
npm install -g wscat
|
|
|
|
# Connect to the server
|
|
wscat -c ws://localhost:8765
|
|
```
|
|
|
|
### Test with curl (websocat)
|
|
|
|
```bash
|
|
# Install websocat
|
|
# Ubuntu: sudo apt install websocat
|
|
# macOS: brew install websocat
|
|
|
|
# Send a ping
|
|
echo '{"type": "ping"}' | websocat ws://localhost:8765
|
|
# Response: {"type": "pong"}
|
|
|
|
# Send a voice command
|
|
echo '{"type": "command", "text": "sube"}' | websocat ws://localhost:8765
|
|
|
|
# Request scene refresh
|
|
echo '{"type": "scene_refresh"}' | websocat ws://localhost:8765
|
|
```
|
|
|
|
### Test with Python
|
|
|
|
```python
|
|
import asyncio
|
|
import websockets
|
|
import json
|
|
|
|
async def test_iobridge():
|
|
uri = "ws://localhost:8765"
|
|
async with websockets.connect(uri) as ws:
|
|
# Test ping
|
|
await ws.send(json.dumps({"type": "ping"}))
|
|
response = await ws.recv()
|
|
print(f"Ping response: {response}")
|
|
|
|
# Send command
|
|
await ws.send(json.dumps({
|
|
"type": "command",
|
|
"text": "agarra el cubo rojo"
|
|
}))
|
|
|
|
# Listen for responses
|
|
async for message in ws:
|
|
data = json.loads(message)
|
|
print(f"Received: {data}")
|
|
|
|
asyncio.run(test_iobridge())
|
|
```
|
|
|
|
### Test with curl (HTTP upgrade not supported directly)
|
|
|
|
Since WebSocket requires an upgrade handshake, use this shell script:
|
|
|
|
```bash
|
|
#!/bin/bash
|
|
# test_iobridge.sh
|
|
|
|
# Using websocat for interactive testing
|
|
websocat ws://localhost:8765 <<EOF
|
|
{"type": "ping"}
|
|
{"type": "command", "text": "sube"}
|
|
{"type": "scene_refresh"}
|
|
EOF
|
|
```
|
|
|
|
## WebSocket Message Types
|
|
|
|
### Client -> Server
|
|
|
|
**Command (voice input)**
|
|
```json
|
|
{"type": "command", "text": "agarra el cubo rojo"}
|
|
```
|
|
|
|
**Ping (health check)**
|
|
```json
|
|
{"type": "ping"}
|
|
```
|
|
Response: `{"type": "pong"}`
|
|
|
|
**Scene Refresh**
|
|
```json
|
|
{"type": "scene_refresh"}
|
|
```
|
|
|
|
### Server -> Client (Broadcasts)
|
|
|
|
**Command Response**
|
|
```json
|
|
{
|
|
"type": "response",
|
|
"text": "Ok, voy a tomar",
|
|
"status": "ok"
|
|
}
|
|
```
|
|
|
|
**Scene Update**
|
|
```json
|
|
{
|
|
"type": "scene_updated",
|
|
"objects": [
|
|
{
|
|
"object_type": "cubo",
|
|
"color": "rojo",
|
|
"size": "big",
|
|
"position_mm": [150.0, 200.0, 280.0],
|
|
"source": "detection"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
## Dora Dataflow Configuration
|
|
|
|
```yaml
|
|
nodes:
|
|
- id: iobridge
|
|
build: pip install -e ./dora_iobridge
|
|
path: dora_iobridge
|
|
inputs:
|
|
voice_out: voice_control/voice_out
|
|
scene_update: voice_control/scene_update
|
|
outputs:
|
|
- voice_in
|
|
env:
|
|
VOICE_HOST: "0.0.0.0"
|
|
VOICE_PORT: "8765"
|
|
```
|
|
|
|
```bash
|
|
dora up
|
|
dora start dataflow.yml
|
|
```
|
|
|
|
## Dependencies
|
|
|
|
- dora-rs >= 0.3.9
|
|
- pyarrow >= 12.0.0
|
|
- websockets >= 12.0
|