Reverse engineering Lenkeng HDMI over IP extender

Short time ago, I was searching for a way how to get HDMI output from camera to PC (and then stream on the Internet). There are PCI-x HDMI input cards on the market, but they cost 100$+. Suddenly, I have found a device, which transmitted HDMI signals over IP network for half of the above price so I took the chance. Specification said something about MJPEG so I thought it might be possible.

Beginning

When the package arrived, first thing was to test if it really works as described and it really did – audio and video output from DVD player transmitted through common ethernet switch to my TV. Second thing I did was of course starting wireshark and sniffing the data.

Screenshot from 2014-01-24 07:46:25

I found out, that it uses multicast to two UDP ports (2068 and 2066) and time to time broadcast some control packets.

Extracting video frames

Highest bitrate was on port 2068 what indicated it is video stream. After a while I found a packet with JFIF header (on picture above) – great! data is not encrypted, nor compressed and contains JPEGs. After investigating packets in sequence, I found out following structure in UDP payload:

| 2B – frame number | 2B – frame chunk number | data |
* frame number – (unsigned int, big endian) all chunks within  one JPEG have same frame number, increments by 0×01
* frame chunk number – (unsigned int, big endian) first image chunk is 0×0000, increments by 0×01, last chunk has MSB set to 1

So I searched google for python multicast listener and modified it to save data to .jpeg files according rules above. After running, I was able to see some images with correct size but they were somehow damaged. First few pixels were OK but then there was a mess, like something was missing. Few hours I elaborated with capturing script what could be wrong but then I’ve got an idea: why does wireshark shows malformed packets? After looking on wireshark packets again, I found out, there are some bytes in the end of packet not part of UDP payload.

Screenshot from 2014-01-24 08:19:15

So I googled raw socket listener (thanks Silver Moon) and manually parsed IP and UDP headers to match correct packets and extract UDP payload with trailing bytes any voila! it worked, I’ve got correct JPEG frames.

#!/usr/bin/python

#Packet sniffer in python
#For Linux - Sniffs all incoming and outgoing packets :)
#Silver Moon (m00n.silv3r@gmail.com)
#modified by danman

import socket, sys
from struct import *
import struct

#Convert a string of 6 characters of ethernet address into a dash separated hex string
def eth_addr (a) :
  b = "%.2x:%.2x:%.2x:%.2x:%.2x:%.2x" % (ord(a[0]) , ord(a[1]) , ord(a[2]), ord(a[3]), ord(a[4]) , ord(a[5]))
  return b

#create a AF_PACKET type raw socket (thats basically packet level)
#define ETH_P_ALL    0x0003          /* Every packet (be careful!!!) */
try:
    s = socket.socket( socket.AF_PACKET , socket.SOCK_RAW , socket.ntohs(0x0003))
except socket.error , msg:
    print 'Socket could not be created. Error Code : ' + str(msg[0]) + ' Message ' + msg[1]
    sys.exit()

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
#sock.bind(('', 2068))
# wrong: mreq = struct.pack("sl", socket.inet_aton("224.51.105.104"), socket.INADDR_ANY)
#mreq = struct.pack("=4sl", socket.inet_aton("226.2.2.2"), socket.INADDR_ANY)
#sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)

sender="000b78006001".decode("hex")
notopen=1
# = open('/tmp/fifo', 'w')

# receive a packet
while True:
    packet = s.recvfrom(65565)

    #packet string from tuple
    packet = packet[0]

    #parse ethernet header
    eth_length = 14

    eth_header = packet[:eth_length]
    eth = unpack('!6s6sH' , eth_header)
    eth_protocol = socket.ntohs(eth[2])

    if (packet[6:12]==sender) & (eth_protocol == 8) :

        #Parse IP header
        #take first 20 characters for the ip header
        ip_header = packet[eth_length:20+eth_length]

        #now unpack them :)
        iph = unpack('!BBHHHBBH4s4s' , ip_header)

        version_ihl = iph[0]
        version = version_ihl >> 4
        ihl = version_ihl & 0xF

        iph_length = ihl * 4

        ttl = iph[5]
        protocol = iph[6]
        s_addr = socket.inet_ntoa(iph[8]);
        d_addr = socket.inet_ntoa(iph[9]);

        #UDP packets
        if protocol == 17 :
            u = iph_length + eth_length
            udph_length = 8
            udp_header = packet[u:u+8]

            #now unpack them :)
            udph = unpack('!HHHH' , udp_header)

            source_port = udph[0]
            dest_port = udph[1]
            length = udph[2]
            checksum = udph[3]

            #get data from the packet
            h_size = eth_length + iph_length + udph_length
            data = packet[h_size:]

            if (dest_port==2068):
              frame_n=ord(data[0])*256+ord(data[1])
              part=ord(data[3])
              print "frame",frame_n,"part",part, "len",len(data),"end?",end
              if (part==0) & notopen:
                 f = open('files/'+str(frame_n)+"_"+str(part).zfill(3)+'.jpg', 'w')
                 notopen=0
              if notopen==0:
                  f.write(data[4:])

Thank you chinese engineers! Because of wrong length in IP header (1044) I have to listen on raw socket!

Initiating stream

All this was done, when both sender and receiver were plugged into network. I also wanted to be able to use only sender and PC. When I plugged in sender only, no stream was broadcasted so I plugged in also the receiver a captured control frames. Each one control packet from receiver was unique so I took payload from the first one and sent it from PC to IP address of sender and I was successful. The sender started to send stream for a few seconds and then stopped. So I started to send control packets one per second and the stream was playing continuously. Then I experimented with length of payload – I started to send shorter frames and I finally found out, that it only needs a few specific bytes. So the final packet is as follows:
Screenshot from 2014-01-25 12:38:41
unicast to 48689/UDP with payload 0x5446367A600200000000000303010026000000000234C2

Extracting audio

Last part of my research was to extract audio from stream. Again, I used wireshark to sniff and analyse packets.
Screenshot from 2014-01-25 12:44:22
I saved the stream to file as raw data and imported into audacity. After trying few formats for importing I found out, that there is some audio playing when set for Signed 32bit PCM, big-endian, stereo, 48Khz.
Screenshot from 2014-01-25 12:48:49
As you can see on the picture, sound was interfered by regular peaks. After investigating packets, I found that the data is well structured except the beginning of payload starting with some 0×00 and 0×55 (as on picture from wireshark). So I took python multicast listener again and truncated first 16bytes from each packet and streamed to stdout:

#!/usr/bin/python

import socket
import struct
import sys

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(('', 2066))
# wrong: mreq = struct.pack("sl", socket.inet_aton("224.51.105.104"), socket.INADDR_ANY)
mreq = struct.pack("=4sl", socket.inet_aton("226.2.2.2"), socket.INADDR_ANY)
sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)
counter=0

while True:
  packet=sock.recv(10240)
  sys.stdout.write(packet[16:])

I started vlc to listen to streamed audio with following command:

./listen-audio.py | vlc --demux=rawaud --rawaud-channels 2 --rawaud-samplerate 48000 --rawaud-fourcc s32b -

and I’ve got audio playing from VLC.

Hardware

Great magic box! As a curious engineer, I definitely had to know what’s inside this gadget. Here are some photos:

IMG_20131123_123422

  • LK-3080-ACL – main processor from Lenkeng
  • IT6604E – HDMI input interface doing HDCP decoding

IMG_20131130_123031

  • LK-3080-ACL – main processor from Lenkeng
  • CAT6612CQ – HDMI out interface

Conclusion

I was searching for solution for live video mixing from cameras with HDMI output. They can be far from mixing computer connected via switches and cheap ethernet technology. With this knowledge I have build a prototype for mixing software in Qt separating input video streams by source IP and mixing/switching them. It’s still in beta phase, but you can see it on my github. If you have any questions or suggestions, feel free to let me know in comment.

Quality test (updated 26/01/2014)

As requested by readers, today I did a speed/quality test. I took jpeg 1080×1920 image and let it play on DVD player. You can compare original and transmitted image here:

new-york-new-york-city-manhattan-manhattan-hotel-millennium filename

Transmitter was streaming 1080p@18fps with 90Mbps bitrate. Unfortunately I have no blue-ray player on hand to test original HD content.

Update 2. Feb 2013

Today I have quickly tested 1080 again and it showed up, that quality is not that bad as on pictures above. Please see following picture. The source was computer showing the same picture in fullscreen mode.

filename

59 komentárov pri “Reverse engineering Lenkeng HDMI over IP extender

  1. Spätné upozornenie: rndm(mod) » Reverse Engineering an HDMI Extender

  2. Spätné upozornenie: Reverse Engineering an HDMI Extender - RaspberryPiBoards

  3. Spätné upozornenie: Reverse Engineering an HDMI Extender | Orange Claymore Red Slime

  4. Spätné upozornenie: Reverse Engineering an HDMI Extender — Blog of MPRosa

  5. Congratulations! Amazing project and some nice hackery right there. I was thinking for a long time about disk recorder for HDV camcorders. They use firewire for data transfer. It would be easy if you can find small device with FW port and linux OS but it looks like it’s almost impossible – there are no compact devices, small enough to mount in on the camera.

    Your article inspired me to think differently – why use Firewire if we have HDMI present on almost every camcorder? HDMI recorders are also expensive, even HDMI capture cards are out of my price range (not to mention you can’t build small device around it).

    But what if we use one of those HDMI extenders, hook it up to the Raspberry Pi and capture all data to the SD card? It would be simple, small and battery powered – perferct for my application.

    From my understanding you get continous audio stream and every video frame as a separate JPEG file, right? It might be possible to use ffmpeg to compress it with h.264 and write video and data together as a single file (h.264 or maybe less lossy format).

      • For my application image quality is on the first place. I see there is huge quality loss but it might be caused by DVD player (note moire patterns and serration on “teeth” on the edges, typical to scaling down by line skipping). Do you have other device with HDMI port? Some camcorder so you can record footage by the camcorder itself and via HDMI extender then compare.

    • Thank you, I haven’t tried continuous streaming for such a long time but when I left it conected and powered on for a week, it retained fully functional.

    • I have used usb to ttl serial converter but I was not able to cummunicate without errors so I gave up. I’ve got some output but the communication was somehow crippled. Maybe I’ll try again later with some other converter. So far I have found it is some sort of debug console.

      • Show me such communication. Or try again usign a proper shield and SHORT cables. Usual speed is 57600 8n1 or 115200 8n1 but there can be variations.

        • I have received the device as well and hooked up my jtagulator to the device.
          It’s
          1 = +
          2 = Rx
          3 = Tx
          4 = Gnd
          using 3.3v

          When I send an enter \r\n it sends back
          Unknown Command!!
          TFCMD>

          So I am looking into that currently.

        • Typing help gave me an item.

          reg : Read/Write reg. of taifa device

          sys : Do some system set

          spi : Read/Write SPI device

          osd : Read/Write SPI device

          gpio : GPIO port Read/Write

          mdio : Read/Write mdio device

          i2c : Read/Write i2c device

          hdmi : Read/Write HdmiTx device

          hdmi : Read/Write HdmiRX device

          dbg : Debug message level

          ch : Audio buffer control

          i2cm : Read/Write i2c device with multi register address

          edid : EDID Read/Write

          3 : Do EP9x53 controller Task

          i2s : T/R audio by I2S mode

          l2 : Set L2 routing table.

          l4 : Set L4 routing table.

          sdram : External SDRAM Test

          cap : Capturer fskp, CRC, bypassSD, shsize, svsize

          ver : Debug Console Version Information

          av : select Tx/Rx format

          q : video quality toggle

          q : show stack status

          fps : show/off fps for test

          inband : inband packet test

          TFCMD>

          • the “av” command is very important.

            what options does it have?

            probably changing the format is the easier route to follow.

        • Boot sequence:0123456
          *************************************************
          * Initializeing MM *
          *************************************************
          Start:0×00100000 End:0×00280000, Size:0×00180000(1536K), Pages:384

          TF680 FW Verion: Tx_v1.07_C904 ITE 152MHz IR LAN_MODULE
          FW Release Date: Nov 2 2012, 11:04:55
          TF680 MAC: 00:0B:78:
          TF680 IP: 192.168.168.55
          All of Tasks Created
          [taifa Debug Console]
          (c)2010 taifatech inc. All rights reserved.
          **Type ‘help’ for more details; ‘exit’ to quit**
          TFCMD>
          OtoM TFVEP is disconnected !!!

        • So here are some commands and their results.
          TFCMD>reg
          WARNING: too few parameters…
          USAGE:
          reg r 90907000 10: read sequentially from reg 0×90907000 len.0×10
          reg w 90907000 aa55aa55 55aa55aa: write 0xaa55aa55, 0x55aa55aa to reg 0×90907000.
          reg f 90907000 aa55aa55 20: write 0xaa55aa55 from reg 0×90907000 to reg 0x9090701f
          reg e : enable/disable message output of wirte operation
          TFCMD>edid
          WARNING: too few parameters…
          USAGE:
          edid r 2 : Read EDID content of HDMI Port 2
          edid w 2 5 : Write EDID 5th content to EPROM for HDMI Port 2
          HDMI EDID : PortI – 5th; PortII – 6th
          TFCMD>sdram
          WARNING: Unknown Option…
          USAGE:
          sdram d startAddr endAddr: SDRAM Data Bus Test
          sdram v baseAddr byteLen: SDRAM Device Test
          sdram s startAddr endAddr: Scan All SDRAM Test
          TFCMD>ver
          Debug Console Version Information
          version:0.2
          TFCMD>av
          WARNING: too few parameters…
          USAGE:
          (for demo only)
          av [video mode=>1:1080p, 2:720p]
          3:960×600
          4:960×544
          5:960×536(1/2 scale down)
          6:1267×728(2/3 scale down)
          7:1440×1080(source 1080p 3/4 scale down)
          8:960×544(source 720p 3/4 scale down)
          9:JPEG Encoder 420 format
          TFCMD>q
          q p [n]: paper quality)
          q d [n]: default quality)
          n: 0 ~ 24(18h), 0 is best, and 24 is worse
          TFCMD>fps
          WARNING: Unknown Option…
          USAGE:
          fps show : show fps
          fps off : off fps
          TFCMD>inband
          WARNING: Unknown Option…
          USAGE:
          send : send test data
          TFCMD>

          • I have see this. I’d be great if they send us technical datasheet. I asked them but it looks like they don’t won’t to release such documents.

          • Command “q” is what we should modify. Did you tried to set it for the best quality? Can you post some sample footage?

          • I took some time to find some sort of service manual for these devices. It looks like SoC is Taifatech TF-680. I couldn’t find any helpful files tho. It looks like these devices have Web control panel so type its IP address to the web browser.

          • It has web interface but it allows only firwmare upgrade, EDID info upload and IP change.

          • Not yet, I haven’t even connected an hdmi cable or network cable.

            I will not have time today to do so, but I could do that tomorrow evening and post some results.

          • I’d be thankful for checking what real quality looks like and see if “q” command actually does something. I’m about to order this device myself but I must be sure the quality meets standards for my application.

          • I have see this. I’d be great if they send us technical datasheet. I asked them but it looks like they don’t won’t to release such documents.

  6. To complete your work you should create 2 stand-alone programs: one to receive (i.e. record) and one to send (i.e. playback) video streams with optional conversion to mpeg :D

    • Good suggestion but I’m too lazy to do things I do not need :D . I can sell you the reciever and you can continue with the second part :)

      • LOL! I’m lazier than you :)
        After inspecting the device I think it’s even expensive for the low quality of the streams it outputs.

  7. Spätné upozornenie: Reverse engineering Lenkeng HDMI over IP extend...

  8. There’s an open question being debated on Hackaday about bandwidth vs. quality of these devices. Given that it’s only a 100Mbps port and using MJPEG, there is going to be a definite limit on quality.

    Are you able to provide some bandwidth-usage number (i.e. how close to the max 100Mbps does it get?) as well as some 1080 screenshots of “clean” content, e.g. from a Blu-ray player. That would also answer the HDCP question too ;-)

    Thanks!

      • There does seem to be some fairly significant loss of quality in your pictures.

        Those who want high quality will probably do better when a high quality HDMI capture card.

        That said, this does a very good job considering it’s 1/2 the price and gives you a free receiver to view the content as well.

        • I’d hold on expressing opinions on image quality. Look carefully on the picture. It looks like the image was scaled down and than scaled up. Maybe his DVD player is doing that conversion. Let’s wait for sample footage from other video source.

          My advice is to use camcorder with HDMI out to see what quality camcorder records and what quality can be achieved via HDMI capture. Another good device to test would be a PC with HDMI on the GPU: record computer screen via HDMI extender and do a print screen and then compare.

  9. Where can i buy one of this?
    Does it have the model / manufacturer?
    Is it available in something with easy international shipping like dx.com aliexpress other than ebay?

  10. Interested in knowing if the unit streams a constant framerate regardless of the input resolution or does it use the input resolution to determine the framerate?

    I am involved with running conferences and these would be perfect if we could feed the MJPEG stream into something like GStreamer so we can feed it into our mixing software.

    • There is an option to overlay the FPS with the serial console, I will overlay this Wednesday and see if it is constant or fluctuating too much.

      My goal is to record it and replay it over an blackmagic device.

      What I also saw was that it de-interlaces it, so 1080i 5994 in, will result in 1080p60 probably.

    • The best HDMI capture device I have ever used was made by Blackmagic, I used the H.264 pro (which is £325) for my work but their other kit is also good. If you can use PCIe then their capture cards are well priced. The problem is that HDCP doesn’t work on these cards.

      I’ve bought one of these kits to see how they perform for myself.

  11. Great! The quality loss is not that significant, exactly as I predicted :) I’m curious if it can be even better if we play a little bit with the console.

    Manufacturer replied to my first message but then they stopped. I asked them about SDK which would allow us to write new firmware, possibly with USB support (TF-680 has USB host!) for WiFi dongles which would give us better transfer and maybe better quality. It has many more cool features but without SDK we aren’t going to make anything…

    Do you have ideas how we can obtain one?

  12. Great beat ! I wish to apprentice while you amend your site, how could i subscribe for a blog website? The account aided me a acceptable deal. I had been tiny bit acquainted of this your broadcast offered bright clear concept

  13. Thinking further on this, why don’t you convert your HDMI receiver into a gstreamer source? Then you can just build your switcher/matrix/mixer around gstreamer and then you have flexibility for your sources? It seems building in the hardware to your design would limit it?

    • I could, but I want to have complete controll on all parts so I can do some optimizations. Next thing is, that I wanted to avoid any external utilities which can introduce unwanted buffering/latency.

  14. So, I did some research on the unit this week during the evenings and this morning.
    In short: Don’t buy it. Not that reliable.

    In long:
    I used the following equipment:
    -Sony BDPBX39 Blu-ray player outputting at 1080p60
    -Philips 22pfl4505d/f7 – TV – Maximum input 1080i60

    Audio quality:
    -Audio is in good quality, as stated before, most people won’t hear the difference, but it is 2 channels only, so if you have this extender in between and do want to do multi channel 5.1 surround sound you will unable to do this.

    Video quality:
    -It is OK for JPG, encoding quality seems OK, when in high detail shots you see the degradation but meh.
    -It does do de-interlacing on chip, so if you have an interlaced signal, you will get an de-interlaced signal into the receiver instead of just sending top or bottom field
    -It appears to always downgrade to a maximum of 30 FPS (See Network for more details about FPS)
    -The Rx end downgraded to 720p to make an progressive resolution to not deal with interlacing.

    Audio/Video synchronization:
    -BAD, it is easy visible that the audio is out of synchronization and never appears to be corrected. In the capture program you would need the ability to sync up audio / video.

    Network quality with NetGear GS105
    -Test environment where it goes just from Tx –> Netgear –> Rx, no other network devices attached to the NetGear using 3 feet Cat6 cable
    -There are a lot of dropped frames, it rarely gets to 30 fps, but is mainly arround 13-17fps, often it drops to 0 fps and only audio passes through where it stopped completely twice.

    Network with direct Cat6 cable (3 feet)
    -Mainly remains at 30FPS, sometimes there is a drop, where it says 1 has failed, but actually 2 frames are missing.

    Extra research
    -Packet at port 48689 (all values are hex, unless otherwise specified)
    –Receiver sends one when it receives one from the sender.\
    –Endianness is different in the packet counter than in the rest of the packets
    –[0..4] 5 byte header [54:46:36:7a:63]
    –[5] 1 byte define what you are
    —01 = Sender
    —02 = Receiver
    –[6...7] always 00 00
    –[8...9] 2 bytes of an internal counter of how many 48689 packages have been send. Overflows at ff ff back to 00 00 to 01 00
    –[10..18] two different sequences for sender / receiver
    —[00:03:03:03:00:24:00:00:00] = Sender
    —[00:03:03:01:00:26:00:00:00] = Receiver
    –[19...22] Uptime receiver in ms (0 for sender)
    All bytes are 0 for the receiver from now on to make a length of 512 bytes
    –[23...26] [00:00:00:00]
    –[27] Some flag byte to indicate the signal
    —03 for signal
    —10 for no signal
    –[28..29] Width of signal (probably input or encoded width)
    –[30..31] Height of signal (probably input or encoded height)
    –[32..33] FPS of signal
    –[34..35] Width of signal (probably input or encoded width)
    –[36..37] Height of signal (probably input or encoded height)
    –[38..39] [00:78]
    –[40..43] Uptime in ms for the sender
    –[44..49] [00:01:00:00:00:00]
    –[50] Some indicater of number of receivers connected but doesn’t change when one disconnects
    —00 when none is connected
    —02 when one is connected
    –[51] [0a]
    For the rest it’s filled with 00′s to pad up to 520 bytes.
    -Packet on port 2067
    –This is a frame count packet.

    The rest of the details are already covered in this blogpost.

  15. Can anyone help me?
    Since I plugged the sender and receiver in the network the Wi-Fi connection of my laptop will not connect to the router anymore.
    Any suggestion?
    Thanks in advance.
    Eric

    • Yes, this is normal if you are using non-multicast capable switches. They distribute multicast as broadcast and overload your wifi. You should use/test these convertors on separate subnet or somehow filter multicast on you wifi AP.

  16. Spätné upozornenie: Notable on the InterWebs Newsletter 2 | CoderZen

  17. Hi Huuf

    I want to use this device just to connect a PC to a long distance monitor (about 40 metres) at 1920x1080p resolution, mainly for powerpoint pesentations, means resolution is important.

    You said that receiver downscales to 720p. Is it true for 1080p input content or just for interlaced ?

    Thanks

  18. Danman,

    The jpeg file you captured at receiver is 737.460 bytes, means 5.9 Mbits

    Then at 30fps will be 177 Mbits/s stream

    So, How can this device broadcast 1080p at 30fps with a 100Mbit lan interface ?
    I’m missing something ?

Pridaj komentár

Vaša e-mailová adresa nebude zverejnená. Vyžadované polia sú označené *

Môžete použiť tieto HTML značky a atribúty: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>