1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

BUG Hyperion.ng - web interface error with more than 400 leds

Discussion in 'Bug Reports' started by Rick164, 12 July 2017.

  1. Rick164

    Rick164 Administrator Staff Member Administrator

    Messages:
    169
    Hardware:
    RPi2, +Arduino, +AtmoOrb
    Hi,

    Been testing Hyperion.ng and it seems to crash whenever I set it up with more than 400 leds (using APA102 600 leds), when opening the interface with that amount will just keep on reporting "Lost connection to Hyperion service!" in a loop.
    Nothing in verbose logs and it does work fine except for the web interface part, easiest to duplicate is with the attached config and used it with latest git version (commit: 622a1718085589da58f58e9f1a7617d940e94ee3).
     

    Attached Files:

    Last edited: 14 July 2017
  2. Rick164

    Rick164 Administrator Staff Member Administrator

    Messages:
    169
    Hardware:
    RPi2, +Arduino, +AtmoOrb
    Seems web config can't read the JSON config correctly as it reports this in firefox web console:

    Code:
    [websocket::onmessage] SyntaxError: JSON.parse: end of data after property value in object at line 1 column 16466 of the JSON data
    When editing it to below 400 leds it works fine, best guess:

    - Buffer size exceeded as it does like json line count related as well
    - JSON config reader on web end only can't handle more than 400 leds
     
    Last edited: 14 July 2017
  3. Brindosch

    Brindosch Administrator Administrator

    Messages:
    637
    Hardware:
    RPi1/Zero, RPi2, RPi3, +nodeMCU/ESP8266
    Greetings @Rick164
    Yet another websocket issue, first it was the config write (you remember) and now it's also reading as your config exceeds a buffer.
    I bet it's the server, because i can add 400 leds and it works fine on 64bit, i increased to 800 and now i have the same issue. You are using a Pi/32bit OS?
    However, there are three solutions.
    • expand websocket handler to handle properly the messages
    • i plan to evaluate soon the tcpSocket upgrade to websocket function which is available since qt5.9. It may be also required to backport some code that it works together with qt5.2.
    • another 3rd party lib to hack it into the webserver
    The second option is my fav, as it is more native and doesn't require additional lib compilation _when_ we reach qt5.9 as min version. To be honest, this might be in 2022 if you keep a "os releases over last 4 years are supported".

    To make sure i am right that a proper websocket server fixes this (I really want to avoid a reinvention of wheel), i plan to implement the qt5.3 websocket server with it's own port for testing. I will keep you updated.

    I am currently working on multi process management, if i am done i will have a look at this.

    Have a nice weekend,
    brindosch

    PS: As i implemented the json-rpc at http level it is a higher interest to move the config write again to the json-rpc itself instead webserver only. It was a subsequent hack of the websocket issues

    Edit2: There is no problem with http jsonrpc, it all comes down to websocket server
     
    Last edited: 15 July 2017
    • Thank you Thank you x 1
  4. Paulchen-Panther

    Paulchen-Panther Ich komme wieder, keine Frage! Developer

    Messages:
    89
    Hardware:
    RPi1/Zero, 32/64bit, +Arduino
  5. Brindosch

    Brindosch Administrator Administrator

    Messages:
    637
    Hardware:
    RPi1/Zero, RPi2, RPi3, +nodeMCU/ESP8266
    Ja und nein, das läuft jetzt genau anders herum. https://github.com/hyperion-project...bsrc/jsonserver/JsonClientConnection.cpp#L212
    Der Server schiebt Daten, die nicht komplett gelesen werden können, sei es wegen dem mask oder anderem. Wenn ich das richtig verstanden habe. Das passiert auch wenn zb der base64 string vom image stream größer als 500x500px wird.

    Zum server schreiben
    qtWebsocket erlaubt ein lesen per Frame, aber auch komplett (nehme ich an, steht indirekt drunter -> http://doc.qt.io/qt-5/qwebsocket.html#textFrameReceived) das erspart das zusammensetzen bis der FIN kommt. Momentan wird erwartet, dass der FIN in jeder msg drin steht, tut es das nicht, gibts nen quit. Richtig?

    Senden
    Ich hab nicht nachgesehen was dahinter passiert. Die länge eines QStrings ist drin: http://doc.qt.io/qt-5/qwebsocket.html#sendTextMessage
    Also theoretisch müsste das alle Probleme die man jetzt hat und für die Zukunft (wss) beseitigen. Ich schätze IE11 läuft dann auch nebenbei.

    Praktisch werde ich das testen, wenn ich mit dem hyperiond-starter fertig bin. Ich sehe auch keinen Nutzen den selbst gebastelten mist von 2014(?) zu reparieren :)

    Schönes Wochenende @Paulchen-Panther ,
    brindosch
     
  6. Paulchen-Panther

    Paulchen-Panther Ich komme wieder, keine Frage! Developer

    Messages:
    89
    Hardware:
    RPi1/Zero, 32/64bit, +Arduino
    Ich glaube nicht. Der Server kürzt die Message einfach ab. Die Fehlermeldung von Rick164 sagt ja nicht umsonst "end of data":
    Code:
    [websocket::onmessage] SyntaxError: JSON.parse: end of data after property value in object at line 1 column 16466 of the JSON data
    Hier ein anderer Beispielcode: https://github.com/neeraj9/SignalBl...c1066ae0/http/HttpTcpWebsocketServer.cpp#L104

    Könnte man so sagen, aber das zusammensetzten übernimmt QT für dich. Wenn du den QtWebsocket Server portieren möchtest, wird dir nichts anderes übrig bleiben als mit den FIN bit zu arbeiten.
     
  7. Brindosch

    Brindosch Administrator Administrator

    Messages:
    637
    Hardware:
    RPi1/Zero, RPi2, RPi3, +nodeMCU/ESP8266
    Ich hab mal im source code gewühlt. Frame zerhacker ist da -> https://github.com/qt/qtwebsockets/blob/5.9/src/websockets/qwebsocket_p.cpp#L753

    Ich schätze die Wahl hat man so oder so nicht, der spec will es so. Eigentlich will ich mit WebSocketServer::handleConnection(TcpSocket) nen upgrade machen. Allerdings ist mir das etwas schleierhaft wie das gehen soll. In dem Moment in dem du feststellst, dass der request nen handshake ist, läuft der parsing Vorgang schon und handleConnection() setzt direkt bei nextPendingConnection() an um dann auf den handshake zu warten. Vielleicht sehe ich auch den springenden Punkt nicht und die Lücke füllt sich von allein mit einer Wiederholung oder ähnlichem.

    Natürlich könnte man die betreffenden codezeilen auch kopieren um eine Verarbeitung von und zu frames zu haben. Wäre eine kurzfristige Lösung des Problems.

    @Rick164
    Depending on available time a quick solution will be to copy the frame creation and extraction for proper writing and reading for websockets. Our implementation is simply incomplete. The long runner here is together with ssl support the use of qts own websocket implementation because it's already done and i don't want to take care of future updates for such lowlvl stuff.
     
    • Like Like x 1
  8. Rick164

    Rick164 Administrator Staff Member Administrator

    Messages:
    169
    Hardware:
    RPi2, +Arduino, +AtmoOrb
    Ah I see, German is a bit rusty here so if I understand correctly in future updates this will be handled better due to a more recent qt websocket implementation?

    At the moment this is fine though and is pretty much a rare case to have this amount of leds anyway so no rush, can live without the web interface and do the config edition manually like with V1 :)
    Running Rpi 3 (64-bit) but can also test on earlier models if needed.
     
  9. Brindosch

    Brindosch Administrator Administrator

    Messages:
    637
    Hardware:
    RPi1/Zero, RPi2, RPi3, +nodeMCU/ESP8266
    Sorry for that,
    i want to resolve it rather now than later, the webconfig write should move again to the json-rpc processing, as it is currently just possible via webserver. So it is not available for tcpsockets and websockets which is just a bad design and as i implemented http json-rpc support it is more than welcome to make it that way available to all communication ways now and in future (udp for example).
     
  10. Rick164

    Rick164 Administrator Staff Member Administrator

    Messages:
    169
    Hardware:
    RPi2, +Arduino, +AtmoOrb
    No worries, my C++ skills are mediocre but let me know if you need a hand as we can probably brainstorm :)
     
  11. Brindosch

    Brindosch Administrator Administrator

    Messages:
    637
    Hardware:
    RPi1/Zero, RPi2, RPi3, +nodeMCU/ESP8266