1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

REQUEST Android Grabber App

Discussion in 'Feature Requests' started by Brindosch, 10 May 2016.

  1. jeckyll

    jeckyll New Member

    Messages:
    21
    Hardware:
    RPi1/Zero, RPi2, RPi3, 32/64bit, +Arduino
    Currently i got no netflix account, but i found this:

    http://www.vysor.io/

    As it's just a Chrome Addon, it maybe works on a rpi3 with Pixel Desktop and Chrome Browser.
    So it would be 1: Nvidia Shield-->Raspberry-->Ambilight
     
    Last edited: 1 February 2017
  2. Angrysalad

    Angrysalad New Member

    Messages:
    5
    I dont think the chrome addon will work. At least other addons didnt work on chromium since the NPAPI interface is removed. I think i tested Vysor too.

    When i get home from work later this week i'll try capturing the screen trough ADB with the internal screengrabber as sugested earlier in this thread. With this no addon is needed and i think we can set the resolution and framerate via the command line. This will require the shield to be connected to the rpi with usb.
     
  3. jeckyll

    jeckyll New Member

    Messages:
    21
    Hardware:
    RPi1/Zero, RPi2, RPi3, 32/64bit, +Arduino
  4. lololo

    lololo New Member

    Messages:
    3
    Hardware:
    RPi2, RPi3, 32/64bit, iMX6, +Arduino, +PhilipsHue
    Hello,

    Are you sure that it does not support acceleration hardware ?!

    Have you tested this parameter:

    <advancedsettings>
    [...]
    <video>
    <usedroidprojectioncapture>true</usedroidprojectioncapture>
    </video>
    [...]
    </advancedsettings>

    https://github.com/koying/SPMC/wiki/How-do-I-get-Hue,-boblight,-hyperion...-to-work-with-Mediacodec-Surface?
     
  5. jeckyll

    jeckyll New Member

    Messages:
    21
    Hardware:
    RPi1/Zero, RPi2, RPi3, 32/64bit, +Arduino
    So here are some testing results:

    On my Laptop with Ubuntu 16.04 i compiled minicap as described here:
    https://github.com/openstf/minicap

    I need to set the resoulution by hand as autodetection didn't worked on my One M8 (change the arg line in run.sh)

    Now i'm able to get the binary output of the screen on terminal via
    nc localhost 1313 in the following format: (taken from Minicap Readme)

    It is assumed that you now have an open connection to the minicap socket. If not, follow the instructions above.

    The minicap protocol is a simple push-based binary protocol. When you first connect to the socket, you get a global header followed by the first frame. The global header will not appear again. More frames keep getting sent until you stop minicap.

    Global header binary format
    Appears once.

    Bytes Length Type Explanation
    0 1 unsigned char Version (currently 1)
    1 1 unsigned char Size of the header (from byte 0)
    2-5 4 uint32 (low endian) Pid of the process
    6-9 4 uint32 (low endian) Real display width in pixels
    10-13 4 uint32 (low endian) Real display height in pixels
    14-17 4 uint32 (low endian) Virtual display width in pixels
    18-21 4 uint32 (low endian) Virtual display height in pixels
    22 1 unsigned char Display orientation
    23 1 unsigned char Quirk bitflags (see below)
    Quirk bitflags
    Currently, the following quirks may be reported:

    Value Name Explanation
    1 QUIRK_DUMB Frames will get sent even if there are no changes from the previous frame. Informative, doesn't require any actions on your part. You can limit the capture rate by reading frame data slower in your own code if you wish.
    2 QUIRK_ALWAYS_UPRIGHT The frame will always be in upright orientation regardless of the device orientation. This needs to be taken into account when rendering the image.
    4 QUIRK_TEAR Frame tear might be visible. Informative, no action required. Neither of our current two methods exhibit this behavior.
    Frame binary format
    Appears a potentially unlimited number of times.

    Bytes Length Type Explanation
    0-3 4 uint32 (low endian) Frame size in bytes (=n)
    4-(n+4) n unsigned char[] Frame in JPG format



    Seems very fast and also works on Amazon video app and every app i tried

    So i think what we need is:

    A hyperion module which utilizes minicap for getting the Screen Data from an Android device an send the output to hyperion, as minicap is already Android 7.1 compatible this should give us a good way of capturing the screen on many devices.

    The great is that it works without root an that frames are only send if there are changes, no changes no new frames

    AFAIK this would be the only possible method for getting screen data from Android devices since local installed apps won't have access to the screen data (except you're rooted)

    Now the bad, i'm no developer so i'm unable to do by myself, but i'm allways willing to help, test or debug



    So, why minicap?
    The Android own screencap function for adb is way too slow, we would only get 1 frame every few seconds, not enough for hyperion. And why not use an existing method if available?

    My Shield pro arrives tomorrow and then i check if it also works
     
    Last edited: 7 February 2017
    • Like Like x 1
  6. Brindosch

    Brindosch Administrator Administrator

    Messages:
    679
    Hardware:
    RPi1/Zero, RPi2, RPi3, +nodeMCU/ESP8266
    Are you using Android 5.0 or higher?
    If so it uses MediaProjection https://developer.android.com/reference/android/media/projection/MediaProjection.html
    Sounds promissing, we don't need more than 10 fps anyway.

    @BioHaZard1
    I think you could give it a go.
    The only issue is see here is the api lvl. As you can't request permissions during runtime until 6.0 not sure how it would be with a Android 4.x app to get access or even the api for something which is available with android 5.0.

    Maybe it's much better to push this in a capture app. As i read the adb access from an app is limited, but no source to what. A version for Android 5.0 with MediaProjection and (if working) one with adb for 4.4 and lower..
     
    • Like Like x 1
  7. jeckyll

    jeckyll New Member

    Messages:
    21
    Hardware:
    RPi1/Zero, RPi2, RPi3, 32/64bit, +Arduino
    I currently use Android 6.0.1
     
  8. TPmodding

    TPmodding Administrator Staff Member Administrator

    Messages:
    911
    Hardware:
    RPi1/Zero, RPi2, RPi3, +Arduino
    are you able to test it with netlix? thats what most people looking for...and netflix blocks more than amazon video

    but sounds great everything :)
     
  9. jeckyll

    jeckyll New Member

    Messages:
    21
    Hardware:
    RPi1/Zero, RPi2, RPi3, 32/64bit, +Arduino
    I can check with netflix, but i need the device from a friend, cause i got no account, but the app itself works
     
    • Like Like x 1
  10. jeckyll

    jeckyll New Member

    Messages:
    21
    Hardware:
    RPi1/Zero, RPi2, RPi3, 32/64bit, +Arduino
    Netflix works also, i will post a video i took of the capture so you can see the speed (i did with stf which is also based on minicap)

    Netflix:


    Amazon:


    Don't know about 4k but i think 1080p is better than nothing and would fit for most people
     
    Last edited: 7 February 2017
    • Like Like x 1
  11. b1rdhous3

    b1rdhous3 Member Developer

    Messages:
    51
    Hardware:
    RPi1/Zero, RPi3, 32/64bit, +PhilipsHue
    This is so great!
    Thank you for your investigations on this. I think this would so many setup problems with usb grabbing. This is a way better solution imho.
     
    • Like Like x 1
  12. TPmodding

    TPmodding Administrator Staff Member Administrator

    Messages:
    911
    Hardware:
    RPi1/Zero, RPi2, RPi3, +Arduino
    Would nice to know if this one works with android tvs :D that would be the ultimative solution :D

    thanks for the videos!
     
  13. jeckyll

    jeckyll New Member

    Messages:
    21
    Hardware:
    RPi1/Zero, RPi2, RPi3, 32/64bit, +Arduino
    it should, the shield for example also has adb on board. In theory it should also work via wireless adb.

    @TPmodding
    You can try going in settings, find the build number and click several times on it. After that you should have a new entry called developer options, there you can enable debugging.
     
    Last edited: 7 February 2017
    • Like Like x 1
  14. jeckyll

    jeckyll New Member

    Messages:
    21
    Hardware:
    RPi1/Zero, RPi2, RPi3, 32/64bit, +Arduino
    For anyone who want's to check if at least adb (android debugging bridge) is working with your target device, here is what you need to do:

    1: You need an android based device
    2: Install the adb drivers and tools
    ,either via https://developer.android.com/studio/index.html or manual (look on xda-developers.com)
    3: enable USB debugging on the device.
    Go to settings, info, software identity, more
    tap the build-number several times fast (8-10 times) go back to settings and click on developer options, tick on USB debugging.

    Connect the device to your pc.

    Open a terminal and type adb shell
    And you should land on shell on the device

    For trying over LAN/WiFi also do:
    adb tcpip 5.5.5.5
    adb connect ip-of-device:5.5.5.5

    If this works please report back with
    -device name
    -software version
    -rooted?
     
  15. TPmodding

    TPmodding Administrator Staff Member Administrator

    Messages:
    911
    Hardware:
    RPi1/Zero, RPi2, RPi3, +Arduino
    adb shell over network works with amazon fire stick...
     
    • Like Like x 1
  16. Hardwarechaos

    Hardwarechaos New Member

    Messages:
    5
    Hi,

    this is exactly what I need for my shieldTV and desperate to try.
    FYI: Never had any android device without ADB possible.
    And just to confirm Shield TV does ADB via Network.

    THX
    Chaos
     
  17. jeckyll

    jeckyll New Member

    Messages:
    21
    Hardware:
    RPi1/Zero, RPi2, RPi3, 32/64bit, +Arduino
    Anyone here got an Android based TV?
     
  18. Hardwarechaos

    Hardwarechaos New Member

    Messages:
    5
    Hi
    I have a Sony KD-65X8505C. Anything you need?

    THX
    Chaos
     
  19. jeckyll

    jeckyll New Member

    Messages:
    21
    Hardware:
    RPi1/Zero, RPi2, RPi3, 32/64bit, +Arduino
    Can you check if it also has adb?
     
  20. Hardwarechaos

    Hardwarechaos New Member

    Messages:
    5
    I know for sure, because I use vysor from time to time via network adb.

    THX
    Chaos