Short time ago, I was searching for a way how to get HDMI output from camera to PC (and then stream on the Internet). There are PCI-x HDMI input cards on the market, but they cost 100$+. Suddenly, I have found a device, which transmitted HDMI signals over IP network for half of the above price so I took the chance. Specification said something about MJPEG so I thought it might be possible.
When the package arrived, first thing was to test if it really works as described and it really did – audio and video output from DVD player transmitted through common ethernet switch to my TV. Second thing I did was of course starting wireshark and sniffing the data.
I found out, that it uses multicast to two UDP ports (2068 and 2066) and time to time broadcast some control packets.
Extracting video frames
Highest bitrate was on port 2068 what indicated it is video stream. After a while I found a packet with JFIF header (on picture above) – great! data is not encrypted, nor compressed and contains JPEGs. After investigating packets in sequence, I found out following structure in UDP payload:
| 2B – frame number | 2B – frame chunk number | data |
* frame number – (unsigned int, big endian) all chunks within one JPEG have same frame number, increments by 0×01
* frame chunk number – (unsigned int, big endian) first image chunk is 0×0000, increments by 0×01, last chunk has MSB set to 1
So I searched google for python multicast listener and modified it to save data to .jpeg files according rules above. After running, I was able to see some images with correct size but they were somehow damaged. First few pixels were OK but then there was a mess, like something was missing. Few hours I elaborated with capturing script what could be wrong but then I’ve got an idea: why does wireshark shows malformed packets? After looking on wireshark packets again, I found out, there are some bytes in the end of packet not part of UDP payload.
So I googled raw socket listener (thanks Silver Moon) and manually parsed IP and UDP headers to match correct packets and extract UDP payload with trailing bytes any voila! it worked, I’ve got correct JPEG frames.
#!/usr/bin/python #Packet sniffer in python #For Linux - Sniffs all incoming and outgoing packets :) #Silver Moon (firstname.lastname@example.org) #modified by danman import socket, sys from struct import * import struct #Convert a string of 6 characters of ethernet address into a dash separated hex string def eth_addr (a) : b = "%.2x:%.2x:%.2x:%.2x:%.2x:%.2x" % (ord(a) , ord(a) , ord(a), ord(a), ord(a) , ord(a)) return b #create a AF_PACKET type raw socket (thats basically packet level) #define ETH_P_ALL 0x0003 /* Every packet (be careful!!!) */ try: s = socket.socket( socket.AF_PACKET , socket.SOCK_RAW , socket.ntohs(0x0003)) except socket.error , msg: print 'Socket could not be created. Error Code : ' + str(msg) + ' Message ' + msg sys.exit() sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) #sock.bind(('', 2068)) # wrong: mreq = struct.pack("sl", socket.inet_aton("184.108.40.206"), socket.INADDR_ANY) #mreq = struct.pack("=4sl", socket.inet_aton("220.127.116.11"), socket.INADDR_ANY) #sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq) sender="000b78006001".decode("hex") notopen=1 # = open('/tmp/fifo', 'w') # receive a packet while True: packet = s.recvfrom(65565) #packet string from tuple packet = packet #parse ethernet header eth_length = 14 eth_header = packet[:eth_length] eth = unpack('!6s6sH' , eth_header) eth_protocol = socket.ntohs(eth) if (packet[6:12]==sender) & (eth_protocol == 8) : #Parse IP header #take first 20 characters for the ip header ip_header = packet[eth_length:20+eth_length] #now unpack them :) iph = unpack('!BBHHHBBH4s4s' , ip_header) version_ihl = iph version = version_ihl >> 4 ihl = version_ihl & 0xF iph_length = ihl * 4 ttl = iph protocol = iph s_addr = socket.inet_ntoa(iph); d_addr = socket.inet_ntoa(iph); #UDP packets if protocol == 17 : u = iph_length + eth_length udph_length = 8 udp_header = packet[u:u+8] #now unpack them :) udph = unpack('!HHHH' , udp_header) source_port = udph dest_port = udph length = udph checksum = udph #get data from the packet h_size = eth_length + iph_length + udph_length data = packet[h_size:] if (dest_port==2068): frame_n=ord(data)*256+ord(data) part=ord(data) print "frame",frame_n,"part",part, "len",len(data),"end?",end if (part==0) & notopen: f = open('files/'+str(frame_n)+"_"+str(part).zfill(3)+'.jpg', 'w') notopen=0 if notopen==0: f.write(data[4:])
Thank you chinese engineers! Because of wrong length in IP header (1044) I have to listen on raw socket!
All this was done, when both sender and receiver were plugged into network. I also wanted to be able to use only sender and PC. When I plugged in sender only, no stream was broadcasted so I plugged in also the receiver a captured control frames. Each one control packet from receiver was unique so I took payload from the first one and sent it from PC to IP address of sender and I was successful. The sender started to send stream for a few seconds and then stopped. So I started to send control packets one per second and the stream was playing continuously. Then I experimented with length of payload – I started to send shorter frames and I finally found out, that it only needs a few specific bytes. So the final packet is as follows:
unicast to 48689/UDP with payload 0x5446367A600200000000000303010026000000000234C2
Last part of my research was to extract audio from stream. Again, I used wireshark to sniff and analyse packets.
I saved the stream to file as raw data and imported into audacity. After trying few formats for importing I found out, that there is some audio playing when set for Signed 32bit PCM, big-endian, stereo, 48Khz.
As you can see on the picture, sound was interfered by regular peaks. After investigating packets, I found that the data is well structured except the beginning of payload starting with some 0×00 and 0×55 (as on picture from wireshark). So I took python multicast listener again and truncated first 16bytes from each packet and streamed to stdout:
#!/usr/bin/python import socket import struct import sys sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) sock.bind(('', 2066)) # wrong: mreq = struct.pack("sl", socket.inet_aton("18.104.22.168"), socket.INADDR_ANY) mreq = struct.pack("=4sl", socket.inet_aton("22.214.171.124"), socket.INADDR_ANY) sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq) counter=0 while True: packet=sock.recv(10240) sys.stdout.write(packet[16:])
I started vlc to listen to streamed audio with following command:
./listen-audio.py | vlc --demux=rawaud --rawaud-channels 2 --rawaud-samplerate 48000 --rawaud-fourcc s32b -
and I’ve got audio playing from VLC.
Great magic box! As a curious engineer, I definitely had to know what’s inside this gadget. Here are some photos:
- LK-3080-ACL – main processor from Lenkeng
- IT6604E – HDMI input interface doing HDCP decoding
- LK-3080-ACL – main processor from Lenkeng
- CAT6612CQ – HDMI out interface
I was searching for solution for live video mixing from cameras with HDMI output. They can be far from mixing computer connected via switches and cheap ethernet technology. With this knowledge I have build a prototype for mixing software in Qt separating input video streams by source IP and mixing/switching them. It’s still in beta phase, but you can see it on my github. If you have any questions or suggestions, feel free to let me know in comment.
Quality test (updated 26/01/2014)
As requested by readers, today I did a speed/quality test. I took jpeg 1080×1920 image and let it play on DVD player. You can compare original and transmitted image here:
Transmitter was streaming 1080p@18fps with 90Mbps bitrate. Unfortunately I have no blue-ray player on hand to test original HD content.
Update 2. Feb 2013
Today I have quickly tested 1080 again and it showed up, that quality is not that bad as on pictures above. Please see following picture. The source was computer showing the same picture in fullscreen mode.