HDMI to CSI-2 (instead of USB Grabber) - my findings and some technical documentation

My Setup
I build my first Hyperion setup probably 2 years ago. Got a raspberry pi and just connected some sk6812 to it.

These days my setup changed quite a lot, I have wled installed with an esp32 (on the backside of the tv), to control the leds.

And Hyperion still runs on my raspberry pi 4 but in another room, controlling the wled instance.

I have had quite some trouble running HDR/SDR content and thus looked into improvements to make.

Didn't find the perfect solution for that yet, but today(!!!) I played around a bit with a HDMI to CSI-2 bridge.

And I love the improvements the bridge gave me over USB Grabbers. It's way faster than any USB solution I ever tried!

It fells even faster than the setup I have had before, where I didn't run wled.

With USB Grabbers I always felt some delay when using the tv remote (to change something) and the ambilight. It just felt out of sync.

With the HDMI to CSI-2 bridge, I don't feel that! It's amazing Its the same kind of feeling when you have a laggy keyboard. You press some keys, and the input text is delayed by just a bit too much. And then you get a new keyboard and its flipping fast

But everything seems to have a sad side too:
It's completely unstable. Every time the TV turns off and on again, the Bridge needs to be reconfigured as the EDID data for that stream gets lost (?). So I always need to run some commands to set the edid and some timings again.

But even then it's often still not working.

I am currently trying to figure out if the problem is:

- Hyperion

- the source (my HDfury diva or Apple TV)

- v4l2-ctl

- the HDMI to CSI-2 bridge

Technical Side of the HDMI to CSI-2 Adapter

Here a few more infos I found out about the HDMI Adapter/Bridge.

In this forum there are only a few mentions of it. And a few people think it's capped to 1080p with 25fps (or 30fps). Thats not the entire story.

The Chip that's powering the HDMI to csi-2 transformation is the TC358743XBG. The Chip itself supports up to 1080p 60 fps (24 Bits color depth )

There are multiple versions of the Bridge itself floating around to buy.

There are the ones with the 15 Pin CSI-2 cable. These only supports 2 lanes (max of 1080p/30FPS with only 8Bit Color)

And there are boards like the Auvidea B102 (rev2) or the Geekworm C790 which have a 22 Pin (instead of 15 Pin) CSI-2 connector. These support 4 lanes with the full 1080p 60fps 24 bit experience.

But if you look at your raspberry pi, you probably will find a CSI-2 15 Pin connector (for the camera).

And yeah I am sorry to disappoint you, by default the RP (up to 4, I think) can only run the 15 Pin Version...

But the good thing, if you get yourself a computing module and the Compute Module IO Board you get the 22 Pin version.

Or the newest Raspberry Pi 5 also seems to come by default with the 22 Pin Version (4 lanes).

Note: The Zero W also has the 22 Pin connector but only has 2 lanes!!

I myself got the hands on a Waveshare HC100F Bridge (with only 15 pins). I don't know if that's one of the good boards or not.

From what I found it seems Auvidea is one of the biggest sellers, and probably the most reliable one... but that's just from googling.

---

My plan is to get either a raspberry pi 5 or a computing module, and also another HDMI to CSI-2 Bridge and check if I get better reliability out of it.

I want to look a little bit more into how the communication between Raspberry Pi and Source works (over HDMI), and why its not working after every restart.

Also since the adapter supports 24 Bits color depth, wouldn't that mean that the HDFury Diva could be retired, and we could do the HDR to SDR conversion on the pi locally?

As you can see I still have quite a few things to check and work out. But I work and I study full time, so I don't have that much time, but if I have some news I will post them here

I would love to know if the Devs working on the project had the possibility to try such a Bridge? And if I am running into a dead end, or if its worth exploring.

I forgot to add that but here a guide to configure the Bridge.

1080p25 edid:

1080p30 edid:

And here a 1080p50 edid if someone ever needs one.

TPmodding I bought mine for 30€ (Central Europe)

Also: On a lot of websites it says "software limitation 25fps", but thats only when used with raspivid and not with the actual driver and v4t2.

And already an update, after a day of trying around:

It seems there are still some bugs with the latest hdfury diva firmware which gave me some weird behavior with the bridge.

From their discord I got the newest firmware and now the bridge is not completely broken like it was before.

I also tried the bridge directly (no diva) with my PC and my Nintendo switch and it worked!

The only thing that seems to be broken (when using with Hyperion.ng) is that after a restart of the source device (Apple TV, switch, pc, etc.), I always need to go into > Capturing Hardware.

Then disable the USB Capture, and enable it again.

So what I see happening:

- Everything works

- I turn off my Apple TV

- I turn on my Apple TV

- in Hyperion Led Visualization (with Live Video) there is a short flash, and some green. and then Hyperion shows the default off screen. (so it probably thinks device is off?)

Is there any way I can debug the problem, and check why Hyperion won't start screen recording?

(there are no errors in the log output)

are you sure? Because it works when plugging it in. (I can see live video)

Only after restarting the source device ( then hyperion shows the Hyperion logo, so no input), after that I need to reset the hardware capture in Hyperion. Then I can see the live video again.

But my other bridge will arrive next will, will test it with a different one.

yeah tv and bridge are connected to the output of the diva.

But I also tried it with my Pc/Nintendo Switch directly connected to the bridge (without diva). It doesn't matter.

Hear the logs, after I press sleep on my HDMI device.

At timestamp 2024-02-19T20:51:27.913Z you can see I woke up my HDMI device again, and Hyperion noticed that, but directly after that it thinks that the device disconnected (no source left)

2024-02-19T20:51:16.369Z [MUXER|First LED Hardware instance] (DEBUG) (PriorityMuxer.cpp:293:setInputImage()) Priority 240 is now inactive
2024-02-19T20:51:16.370Z [MUXER|First LED Hardware instance] (DEBUG) (PriorityMuxer.cpp:422:updatePriorities()) Set visible priority to 255
2024-02-19T20:51:16.370Z [HYPERION|First LED Hardware instance] (DEBUG) (Hyperion.cpp:645:handleSourceAvailability()) No source left -> Pause output processing and switch LED-Device off
2024-02-19T20:51:27.912Z [MUXER|First LED Hardware instance] (DEBUG) (PriorityMuxer.cpp:422:updatePriorities()) Set visible priority to 240
2024-02-19T20:51:27.913Z [HYPERION|First LED Hardware instance] (DEBUG) (Hyperion.cpp:654:handleSourceAvailability()) new source available -> Resume output processing and switch LED-Device on
2024-02-19T20:51:28.963Z [MUXER|First LED Hardware instance] (DEBUG) (PriorityMuxer.cpp:293:setInputImage()) Priority 240 is now inactive
2024-02-19T20:51:28.963Z [MUXER|First LED Hardware instance] (DEBUG) (PriorityMuxer.cpp:422:updatePriorities()) Set visible priority to 255
2024-02-19T20:51:28.963Z [HYPERION|First LED Hardware instance] (DEBUG) (Hyperion.cpp:645:handleSourceAvailability()) No source left -> Pause output processing and switch LED-Device off

Regarding USB vs HDMI to CSI, I am working with pcs everyday and my feeling for any kind of lag is.... don't know how to describe it. But I notice lag very easily.

My girlfriend first thought "what shitty thing is he doing again, building stuff in our living room". Now she is impressed with Hyperion and the light experience
But she for example doesn't notice any disturbing lag.

But to have a better answer. I did search for my lag tester, but I couldn't find it.

If I get my hands on it again, I will post some numbers