Ongoing: Phantom/Dialog/Spark protocol deciphering and development

[iliapas]

(moved from another thread)

 From initial analysis the unit uses Gstreamer (RTSP?) to stream audio. It runs Linux with 3.x kernel on an ARMV7L-compatible hardware.
 The Spark application starts up some proprietary udp signaling service on port 24242, then Gstreamer service (on port 4040, the standard RTSP port) and populates a database (sqlite-based) with the data available for playback.
 Playback is initiated via some custom-looking protocol, providing the unit the url of the gstreamer stream to play, smth like "http://<hashed address>.dvlt:4040/<song hash><hashed address repeated> if deciphered correctly.
 The unit then issues an http call to the Gstreamer service being run by Spark app on port 4040, doing "HTTP GET" on the url above (the part after the .dvlt:4040)
 User agent is shown as Gstreamer soulhttpsrc libsoup/2.43.1
 From a short analysis of the unit itself, it seems to be running some 3.x version of Linux kernel.
 So there are several parts that are very strange here.
 First, there is no GPL-related information on devialet website, possibly because there is nothing customized in the Linux system it is running. Still, they could possibly be required to provide at least the GPL text with it. IANAL though, especially in French law.
 Second, if the unit understands standard Gstreamer streams, why not provide ability for customers to use standard Gstreamer services for use with the unit?
 In any case, on the port 24242 side, the unit is constantly broadcasting beacon messages with plaintext magic bytes
 "DVL.WHO?" and "DVL.HERE <serial num>".
 When Spark app sees a beacon, it initiates a connection to the unit invoking some custom command with arguments:
 "com.devialet.whatsup.registry.<12hex chars>" and
 "com.devialet.callmemaybe.connect.<32 hex chars>"
 It then continues with second packet sending:
 "com.devialet.getthepartystarted.log-uploader-0.<8hex chars>"
 then url of where to send log (probably):
 "tcp://127.0.0.1:<some port number>"
 and
 "com.devialet.source-0.online-0.authenticated-0.pickupthepieces.<8hex chars>
 and same "tcp://..." string
 At some point the unit sends a packet to Spark, with generic structure of: "<command>.<address> <serial_num> tcp://127.0.0.1:<some_port>".
 Ports are generally different, serial number is the Unit's serial number, and address differs from 8 hex chars (4bytes) to 12 hex chars (6bytes) to 32 hex chars (16 bytes).
 These are possibly to map different service sources/sinks to different ports, and the rest of communication is on those ports.
 As shown above, packets coming from Spark have commands of
 com.devialet.whatsup.registry
 com.devialet.callmemaybe.connect
 com.devialet.getthepartystarted.log-uploader-0
 com.devialet.source-0.online-0.authenticated-0.pickupthepieces
 Commands seen from the unit sent to Spark:
 com.devialet.imaslave4u.configuration-0,
 com.devialet.imaslave4u.soundcontrol-0
 com.devialet.canttouchthis.hid-0.button-0.capacitive-0
 com.devialet.canttouchthis.hid-0.button-0.power-switch-0
 com.devialet.canttouchthis.hid-0.host-remote-0.blue-0
 com.devialet.playthatfunkymusic.configuration-0
 com.devialet.playthatfunkymusic.playback-0
 com.devialet.source-configure-0.bluetooth-0.blue-0
 com.devialet.source-0.live-0.bluetooth-0.blue-0
 com.devialet.source-0.live-0.optical-0
 com.devialet.source-0.online-0.authenticated-0.pickupthepieces (unpause command?)
 com.devialet.source-session-0.online-0.pickupthepieces (unpause command?)
 com.devialet.tiktok.clock-0
 com.devialet.toomanyflows.soundcontrol-0
 com.devialet.toomanyflows.playlist-0
 com.devialet.toomanyflows.playback-0
 com.devialet.toomanyflows.history-0
 com.devialet.toomanyflows.configuration-0
 com.devialet.twerkit.sounddesign-0.toomanyflows-0
 com.devialet.twerkit.sounddesign-0.imaslave4u-0
 com.devialet.masterofpuppets.configuration-0
 com.devialet.getthepartystarted.configuration-0

Playlist synchronization and playback control is done over the playlist port via the gstreamer-rtsp-like urls.
 The sqlite db is in "collection.db" file which is in the user's local app data (/Users/<username>/AppData/Devialet/Spark on win, and probably somewhere in ~/Library/Application Support/ on mac).
 By the way, in Mac OS X El Capitan, the Phantom works very well thru aptx (352Kbit/sec *lossless) at 44/16, as the bluetooth chip in the Phantom is made by CSR, the inventors/buyers/owners of Aptx.
 MacOS doesn't seem to allow rates other than 44KHz though.
 Spark makes some requests to store.dvlt.io (managed by bearstech.com), probably some kind of firmware update check.
 Would be superb if someone with well-configured and working system of 2+ Phantoms would help see the work on the masterofpuppets port.
 There are as it seems 10 ports in use: debug, log, "masterofpuppets", buttons, optical, "getthepartystarted", slave-control, playlist, "playthatfunkymusic", bluetooth.

[xatm094]

Love the command names!

[iliapas]

One thing that's interesting from the first look at the protocol is that it should be possible to create a simple "proxy" tool to use an external Spark server.
It would go like this (capturing UDP only on server-side as rest of communication is via TCP):
1. Have Spark on a server with lots of space and huge music library, with external IP address.
2. Run a small tool to listen for broadcast UDP packets from Spark and publish them somewhere (or decipher announcement protocol).
3. Run a second tool which reannounces the Spark announcements on the local network, with the external IP of the Spark server.
3a. (optional) Capture communication between Phantom and Spark server to block Phantom control from Spark server, as a security measure.
4. Run local Spark and use the remote Spark server music library as if it was local.

Streaming through the standard playback mechanism seems problematic because Phantom seems to download the whole audio file from Spark to play, instead of streaming it..

[iliapas]

Just to give a short update, you can play anything to Phantom by bypasing Spark (taking over ports 4040 and 80).
Following guide is for Windows, but Mac users can do same stuff (even easier) from bash shell. If interested, let know.

You will need netcat (ncat.exe) from NMAP distribution here: https://nmap.org/dist/nmap-7.01-setup.exe

Close Spark, then check that your port 80 is not in use (in cmd.exe run "netstat -an" and check top of the list for anything :80 with status LISTENING).
If you're running Skype, either quit it or go into Advanced settings -> Connection and prohobit it from using ports 80 and 443.

Choose a directory.
Create 4 text files in it: header.txt serve.bat port4040.bat port80.bat 
Copy some music file (FLAC or other type) into the directory and check it's filename.

In header.txt put the text (7 lines) between the ---'s:

----header.txt----
HTTP/1.1 200 OK
Accept-Ranges: bytes
Date: Wed, 23 Dec 2015 15:16:20 GMT
Content-Type: audio/*
Connection: close


---------------------

It's important to leave those 2 blank lines in the text file, as the real header would end with 2 CR+LFs.


In serve.bat put this text (the 3 lines betwee ---'s), changing "09 - Var.flac" to the actual name of the music track you put there:

----serve.bat----
@echo off
type header.txt
type "09 - Var.flac"
-------------------

(If you're wondering, the sample being used here is from album Kveikur by Sigur Ros)


In port80.bat put this one line of text: (or just type in command line)

----port80.bat----
ncat -v -l -k -p 80 -e "serve.bat"
---------------------

In port4040.bat put this one line of text: (or just type in command line)

----port4040.bat----
ncat -v -l -k -p 4040 -e "serve.bat"
------------------------

Now:
1. Close Spark
2. Run (double-click) port80.bat
3. Run (double-click) port4040.bat
4. Start Spark
5. Play any song in Spark.

The file you'll be playing is the file mentioned in serve.bat


The whole thing above is to create a small simple web server, but you can use your own webserver to server the files.
With a simple script, you can set it up to serve files from some directory one by one in alphabetical order, even going into subdirectories.
Or use it to test whether Phantom will understand various file formats, sampling frequencies and bitrates.

[ragwo]

Excellent and exciting news!

[sekriT]

Hey, this is all exciting staff, but I am not sure what you are trying to achieve with all this.  Wouldn't it be better to re-flash the dialog with a music server? There are many linux based music servers out there.

[iliapas]

Hey, this is all exciting staff, but I am not sure what you are trying to achieve with all this.  Wouldn't it be better to re-flash the dialog with a music server? There are many linux based music servers out there.

If you know how to do just that, please do explain how
If you are prepared to open up Phantom/Dialog to connect via JTAG, please let know what you find, or share the existing firmware images of both.

[Gaustabl]

I would really love to be able to controll volume using the irule remote app (wi-fi). That way the phantom would be more available for wife and children....
Best regards Lars

[iliapas]

Hi, Guys!
New development (trivial in hindsight).
Http allows easy redirect, so you can play any online music stream by redirecting Phantom to it.
Example below is some random raw PCM stream from Ibiza

Using the tutorial above, change the header.txt to this (the 8 lines between the ---'s):
----header.txt----
HTTP/1.1 302 Found
Accept-Ranges: bytes
Date: Wed, 23 Dec 2015 15:16:20 GMT
Content-Type: audio/*
Connection: close
Location: http://vp5.net/si/LVN.wav


----end of header.txt--------

Then change serve.bat, removing the last line (serving an actual file).
It should look like this (2 lines between ---'s):
----serve.bat----
@echo off
type header_302.txt
----end of serve.bat----


Then go on as before:
1. Quit Spark
2. Launch port80.bat
3. Launch port4040.bat
4. Launch Spark
5. Play any track. It will instead play the Ibiza PCM stream.

[iliapas]

If anybody is interested in Mac OS version, let know, it's even simpler on Mac
Also, a redirect can be to a dynamic URL which can be processed on server to serve any stream you want, so you can have server-side playlist organizer.