1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

SDR & HDR 1080p/4k capable setup with Hyperion-NG for Media Center

Discussion in 'Hyperion Setup Showcase' started by Awawa, 27 July 2020.

  1. Awawa

    Awawa Member

    Messages:
    42
    Hardware:
    RPi3
    I want to share my configuration for HDR & SDR content and HyperionNG/RaspberryPi3. I made a very long way from utv007 & Rpi3 for SDR years ago. Well, SDR is not a problem but HDR is another story. This solution applies to 1080/4k PC movie player configuration only. Any nonstandard gaming FPS (freesync,gsync et) is not a concern: hardware for that solution doesn't exists yet beside almost analog one: the external capture camera.

    On the market there are only two budget products that can handle it: HDFury x4 (1080 only but it's not so critical for Hyperion usage) and HDFury Diva. You can save yourself money and abandon testing other Chinese matrix/scallers/grabbers that suppose to do the same: they can't at least for now. HDR images that they outputs have bleak colors and are almost useless for our Ambilight experience. Ezcap I use is no better than them....but it allows one hack to make HDR working almost perfectly.

    So you can buy one of HDFury devices or go for compromises: results are almost as good but not quite. And there is a need to output movie always in BT2020 color space (the solution I chose) or to switch configuration manually between HDR/SDR content (otherwise this would results in totally over-saturated colors).

    Used hardware:
    Rpi3, WS2801, ezcap 269, PC mediaplayer (mpc hc, latest madVR, oldie but a goldie myHTPC as frontend) with Flirc for remote control, nvidia gfx, HDR enabled TV/projector.
    The scheme is on the picture below:

    [​IMG]

    USB 2.0 is fully sufficient for Ezcap 269 in terms of speed (thanks to MJPEG compression) and power consumption (well, according to measures it's 0mA so it's probably powered by HDMI line).

    [​IMG]

    USB bus usage for 1080/30 mode is very low even for USB 2.0

    [​IMG]

    Some information about Ezcap 269:

    [​IMG]
    [​IMG]

    There are some significant differences between that grabber and solutions based on analog (or HDMI->analog) UTV007: picture quality and you don't have to set crop for a frame or manually calibrate colors (beside the trick for HDR).

    Sorry for any spelling mistakes: English is not my primary.
     
    Last edited: 29 July 2020
  2. Awawa

    Awawa Member

    Messages:
    42
    Hardware:
    RPi3
    Preparation

    I strongly suggest to install Raspbian with read only mode for a sake of SD card. It also enables switching to write mode if needed.

    The solution is described in the following link but make attention to comments because I had some troubles with /tmp folder and fixing that is a crucial thing (there are some folders hardlinked to /tmp and need to be re-created). Otherwise /tmp remains read-only folder after each reboot.

    https://medium.com/swlh/make-your-raspberry-pi-file-system-read-only-raspbian-buster-c558694de79

    Always verify output format on TV because there is possibility that you can have "almost" very nice output for HDR movie & Hyperion but in fact the content can be transcoded by software/graphic card to SDR output on the fly (these is the least desirable option, because it causes TV picture degradation).
     
    Last edited: 27 July 2020
  3. Awawa

    Awawa Member

    Messages:
    42
    Hardware:
    RPi3
    MadVR configuration

    This results in:
    - automatic hardcoded black bar removal (not necessarily but then we don't need to do it with hyperion and the movie is always stretch to the full screen when MpcHC player is notified by MadVR)
    [​IMG]

    - automatic resolution switching
    [​IMG]

    - and mostly important: output SDR into BT2020 color space (there is an option enabled only for nvidia).
    [​IMG]

    - and same important HDR passthrough
    [​IMG]

    For Ezcap set that in Nvidia driver panel: RGB, 8bit, full range and of course enable HDR in Windows.
    [​IMG]
     
    Last edited: 27 July 2020
  4. Awawa

    Awawa Member

    Messages:
    42
    Hardware:
    RPi3
    RaspberryPi configuration

    SSH to raspbian and create a script in user folder:

    hyperion.sh (make it executable, write-access need to be switched on)
    Code:
    #!/bin/bash
    FILE=/tmp/.hyperion/me
    if [ -f "$FILE" ]; then
        echo "config exists"
    else
        /bin/cp -avr /home/pi/hyperion/. /tmp/.hyperion/
    fi
    /usr/bin/v4l2-ctl --set-ctrl contrast=220;
    /usr/bin/v4l2-ctl --set-ctrl saturation=255;
    /usr/bin/v4l2-ctl --set-ctrl brightness=100;
    /usr/bin/v4l2-ctl --set-ctrl hue=0;
    
    If you don't use read-only mode you can skip that part that checks & copies backup of Hyperion configuration to /tmp folder (there is also symlink from hyperion configuration folder in home to /tmp so Hyperion is using location in /tmp).

    That script "does" the color correction for BT2020/HDR content that was capture by Ezcap.

    I noticed that sometimes after switching between different resolution the kernel or ezcap looses it's brightness&saturation configuration. So I invoke it from cron every few minute and didn't notice any side effects.

    /etc/crontab
    Code:
    ...................
    * *    * * *    your_user    /home/your_user/hyperion.sh
    
     
    Last edited: 27 July 2020
  5. Awawa

    Awawa Member

    Messages:
    42
    Hardware:
    RPi3
    Testing

    1) testing if HDR works OK


    Reference (HDR10-Sony-Bravia-OLED demo):
    [​IMG]

    The content is BT2020&HDR and so TV is reporting, so far co good.
    [​IMG]

    If the script hyperion.sh did not start properly then the grabber outputs bleak colors:
    [​IMG]

    When that script changed saturation/contrast/brightness the result is much better:
    [​IMG]

    2) testing if SDR works correctly

    Without changes made in MadVr SDR content is recognized by the TV as ... SDR or colorspace BT601 ;) That's bad in our case.
    [​IMG]

    Because now with ezcap and settings from hyperion.sh script the result will by totally over saturated:
    [​IMG]

    With MadVR properly set as described in previous post TV recognized SDR as BT2020 movie (notice difference):
    [​IMG]

    And now with hyperion.sh we can process that "SDR" properly:
    [​IMG]
     
    Last edited: 27 July 2020
  6. Awawa

    Awawa Member

    Messages:
    42
    Hardware:
    RPi3
    Tweaks (basic programming skills required)

    With default installation of Hyperion there is some delay between action on TV and Leds.
    For 800x600 and decimation = 1 there is not enough CPU power because Rpi doesn't scale Hyperion on all available cores.
    [​IMG]

    But we can modify code for us to speed up things.
    We dont need 30 FPS which is minimum driver limit but with higher resolution it causes some lags TV vs LEDS. Frame decimation partially solves that problem but results in terrible quality.
    We cant reject an incoming frame but it's light/small one thanks to mjpeg. The process that analyses it is another story. To limit resources it can be changed in software, for example to skip every other from to 15FPS or lower but it requires some changes in hyperion-ng code for now:

    in V4L2Grabbber.h add frame counter:
    Code:
    int _currentFrame;
    and in V4L2Grabbber.cpp modify following method to skip frames
    Code:
    bool V4L2Grabber::process_image(const void *p, int size)
    {
        if ((++_currentFrame)%2 != 0)
            return false;
        else
            _currentFrame=0;

    Another tweak is fast frame copy, it's stable for image capturing for led processing but not quite for the WWW live preview for some reason so beware if you need it (some resolution/decimation breaks it):

    In V4L2Grabbber.cpp replace part that copies image byte by byte
    Code:
    #ifdef HAVE_JPEG_DECODER
            if (_cropLeft>0 || _cropTop>0 || _cropBottom>0 || _cropRight>0)
            {
                QRect rect(_cropLeft, _cropTop, imageFrame.width() - _cropLeft - _cropRight, imageFrame.height() - _cropTop - _cropBottom);
                imageFrame = imageFrame.copy(rect);
            }
            if (_pixelDecimation>1)
                imageFrame = imageFrame.scaled(imageFrame.width() / _pixelDecimation, imageFrame.height() / _pixelDecimation,Qt::KeepAspectRatio);
    
            if ((image.width() != unsigned(imageFrame.width())) || (image.height() != unsigned(imageFrame.height())))
                image.resize(imageFrame.width(), imageFrame.height());
    
            memmove(image.memptr(),imageFrame.bits(),image.width()*image.height()*sizeof(ColorRgb));
        }
        else
    #endif
    Finally result for the same configuration as before (CPU 95%=>33%)

    [​IMG]
     
    Last edited: 29 July 2020
    • Like Like x 1
  7. Awawa

    Awawa Member

    Messages:
    42
    Hardware:
    RPi3
    More advanced tweaks for HDR (under development)

    Alternative for hyperion.sh script: live processing & correction of captured HDR image by Hyperion 2.0.7 with my new mod with imported 3d LUT filter

    Click to enlarge pictures:
    [​IMG] [​IMG]

    [​IMG] [​IMG]

    [​IMG] [​IMG]

    [​IMG] [​IMG]

    [​IMG] [​IMG]

    [​IMG] [​IMG]

    CPU usage is OK, no delay, but had to lower FPS to 10 and set decimation to 2 for 800x600:
    [​IMG]
     
    Last edited: 31 July 2020 at 01:18
    • Like Like x 1
  8. Awawa

    Awawa Member

    Messages:
    42
    Hardware:
    RPi3
    More advanced tweaks for HDR (source code)
    I attached diff patch that adds FPS limit to Hyperion 2.0.7 and 3D LUT tone mapping for HDR to SDR content (and fast frame copy otherwise).

    The idea for 3D LUT algorithm and some code comes from fast-hdr project (https://github.com/so-rose/fast-hdr).

    I made some changes: the original generated 3D LUT file comes in YUV format but Ezcap 269 is using mjpeg RGB format. So lut_rgb.3d file in lut_rgb.3d.zip is now RGB table file to avoid YUV<->RGB converting in real time. I have also expanded color range a bit because it was a little bit to dark.

    Usage:
    1) apply patch and build the release
    2) put lut_rgb.3d from lut_rgb.zip into hardcoded path: /home/pi/.hyperion/lut_rgb.3d (sorry! I think I don't have an access to configuration path from V4L2Grabber, it requires some changes and I want to make that patch simple)
    3) run from build folder bin/hyperiond -d
    Hyperion should find and load 3D LUT file now.
    you should see the following message:
    [​IMG]

    It's only apply to MJPEG (used by ezcap 269 for example) grabber encoding for now! If your grabber is using different encoding (simply YUYV, RGB et) I think it will skip that new part of code.
     

    Attached Files:

    • Like Like x 1
  9. Awawa

    Awawa Member

    Messages:
    42
    Hardware:
    RPi3
    More advanced tweaks for HDR (multithreading processing)

    Need more testing and benchmarks but it seems it's stable. Can be expanded to general frame processing in Hyperion.
    Patch as an attachment (C++11 is required but it's rather not a problem on Rpi).
    Now all cores are involved so we can try higher resolution or FPS.

    Benchmark shows that approach has failed.
     

    Attached Files:

    Last edited: 31 July 2020 at 22:16
  10. TPmodding

    TPmodding Administrator Staff Member Administrator

    Messages:
    1,689
    Hardware:
    RPi1/Zero, RPi2, RPi3, +Arduino, +nodeMCU/ESP8266
    Hey, awesome!
    Is this not possible to add it to hyperion directly? With the a toggle in the WebUI?
     
  11. Awawa

    Awawa Member

    Messages:
    42
    Hardware:
    RPi3
    Thanks! I will try but I'm not promising anything, because that part of WWW interface code is a little bit of magic for me (as QT & C++ that I've forgotten ... too many years developing in c# ;) )
     
    Last edited: 31 July 2020 at 17:28
  12. Awawa

    Awawa Member

    Messages:
    42
    Hardware:
    RPi3
    Good news. It was easier than I thought. No more hardcoded path. The patch as attachment.

    [​IMG]

    [​IMG]
     

    Attached Files:

    Last edited: 1 August 2020 at 23:04
    • Like Like x 1
  13. Paulchen-Panther

    Paulchen-Panther Active Member Staff Member Developer

    Messages:
    718
    Hardware:
    RPi1/Zero, RPi3, 32/64bit, +Arduino
  14. Awawa

    Awawa Member

    Messages:
    42
    Hardware:
    RPi3
  15. Andrew_xXx

    Andrew_xXx Software Developer

    Messages:
    42
    As great research it is, i would not add this to hyperion, its not a real hdr to sdr tone mapping, it just some needlessly resource intensive trick to adjust sdr image that does not contain any hdr data and the resulting images are practically the same as those posted in the 4k hdr grabber thread, frankly the latter are little better, thus, there are no real advantages, its just wasting cpu cycles.

    Calling this option HDR to SDR tone mapping is false and misleading the users, its more like a SDR enhancer.

    It works for that fast-hdr project guy cuz he has direct true HDR video input with all hdr data present, which hyperion doesn't have.

    The thing with HDR is, its way more complicated than it was before, HDR streams have many hdr metadata present, those data needs to be there, at least the mandatory Mastering Display Color Volume metadata (RGB primaries, white point parameters, display maximum and minimum light levels) is needed to calibrate the end tv device, there are also MaxFALL metadata Maximum Frame Average Light Level and MaxCLL metadata Maximum Content Light Level.

    Every hdr device like tvs need these data to display HDR correctly. Every tv has its own HDR screen capabilities that are calibrated in the factory and hardcoded into it, it then calculates the right values for that display from the Mastering Display metadata and other parameters in the hdr stream and its own calibration data.

    It gets even worse, the basic HDR10 format is static metadata for the whole stream, the easiest one, HDR10+ and Dolby VISON is dynamic hdr per scene or even per frame, that metadata is present in SEI headers in HDR video, and it can be a lot of it, it is very complicated to process that data properly.

    To sum it up, its impossible for hyperion to get HDR to do the tone mapping correctly, they only way to do it is to have a grabber that is really outputting raw HDR metadata, which probably doesn't exist, even then im not sure it would be possible due to all the parameters i described. Its a lot of things to do.

    Those parameters are needed for any HDR device, including grabbers, so a working HDR to SDR grabber would already do everything of this and it looks like it would need to have embed its own virtual SDR display parameters to calculate the proper output.

    So those proper grabbers are probably having low MaxFLL and MaxCLL values like most SDR non HDR tvs, but the colours... its always estimated, there is not such thing as exact hdr or sdr colours, with HDR it is always interpreted from HDR values, display or grabber embed parameters, that also means that a single 3d lut file will not work for every hdr video as it can have different properties that needs to be taken into account when tone mapping.

    So, as hard as it gets, we can't win this, too much to calculate and process, raw hdr data unavailable and probably never will be available, its not a hdr to sdr tone mapper, i would not add this, i see no point, but if you want, dont give it this misleading name.
     
  16. Awawa

    Awawa Member

    Messages:
    42
    Hardware:
    RPi3
    OK, I understand that and I repeated many times on this forum that we can't fix HDR from SDR result because some information are lost. As I review fast-hdr there is some math behind these results and they are way better than manipulating with saturation/brightness/contrast (doesn't work with HDR10+ and Dolby Vision either, don't know if there is any windows player that can handle DV) even if it's more resource hungry solution. So there is a tone mapping no doubt but rather "HDR" (broken by the grabber/matrix/splitter/scaller) to SDR - I thought that was clear.
    @Andrew_xXx
    you have alternative in the first post and I mentioned it: HDFury diva or x4 for $$$, other software alternatives are always workaround: that one produces very good result in the real time on Rpi3 and it's first attempt to implement it in Hyperion. You don't have to enable it, it's only an option that you can have or not. Beside if you could read it first before writing I would appreciate it:
    [​IMG][​IMG]
     
    Last edited: 2 August 2020 at 01:19
    • Like Like x 1