Discussion Forum to share and further the development of home control and automation, independent of platforms.

    SmartHome Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Unsolved
    (Last Updated: 23 May 2020)
    • Video Doorbells

      rafale77

      With the boom of video doorbells with the likes of Ring, Skybell and Next doorbell cams, I came to the realization that I did not want to be cloud dependent for this type service for long term reliability, privacy and cost.
      I finally found last year a wifi video doorbell which is cost effective and support RTSP and now ONVIF streaming:

      The RCA HSDB2A which is made by Hikvision and has many clones (EZViz, Nelly, Laview). It has an unusual vertical aspect ratio designed to watch packages delivered on the floor....
      It also runs on 5GHz wifi which is a huge advantage. I have tried running IPCams on 2.4GHz before and it is a complete disaster for your WIFI bandwidth. Using a spectrum analyzer, you will see what I mean. It completely saturates the wifi channels because of the very high IO requirements and is a horrible design. 2.4GHz gets range but is too limited in bandwidth to support any kind of video stream reliably... unless you have a dedicated SSID and channel available for it.

      The video is recorded locally on my NVR. I was able to process the stream from it on Home Assistant to get it to do facial recognition and trigger automations on openLuup like any other IPCams. This requires quite a bit of CPU power to do...
      I also get snapshots through push notifications through pushover on motion like all of my other IPcams. Movement detection is switched on and off by openLuup... based on house mode.

      IPCam
    • Object Recognition

      rafale77

      Sharing a few options for object recognition which then can be used as triggers for home automation.
      My two favorites so far:

      GitHub - asmirnou/watsor: Object detection for video surveillance GitHub - asmirnou/watsor: Object detection for video surveillance

      Object detection for video surveillance. Contribute to asmirnou/watsor development by creating an account on GitHub.

      GitHub - opencv/opencv-python: Automated CI toolchain to produce precompiled opencv-python, opencv-python-headless, opencv-contrib-python and opencv-contrib-python-headless packages. GitHub - opencv/opencv-python: Automated CI toolchain to produce precompiled opencv-python, opencv-python-headless, opencv-contrib-python and opencv-contrib-python-headless packages.

      Automated CI toolchain to produce precompiled opencv-python, opencv-python-headless, opencv-contrib-python and opencv-contrib-python-headless packages. - GitHub - opencv/opencv-python: Automated CI...

      IPCam
    • Facial recognition triggering automation

      rafale77

      I have optimized my facial recognition scheme and discovered a few things:

      My wifi doorbell, the RCA HSDB2, was overloaded by having to provide too many concurrent rtsp streams which was causing the streams themselves to be unreliable:
      Cloud stream
      stream to QNAP NVR
      stream to home assistant (regular)
      stream to home assistant facial recognition.

      I decided to use the proxy function of the QNAP NVR to now only pull 2 streams from the doorbell and have the NVR be the source for home assistant. This stabilized the system quite a bit.

      The second optimization was to find out that by default home assistant processes images every 10s. It made me think that the processing was slow but it turns out that it was just not being triggered frequently enough. I turned it up to 2s and now I have a working automation to trigger an openLuup scene, triggering opening a doorlock with conditionals on house mode and geofence. Now I am looking to offload this processing from the cpu to an intel NCS2 stick so I might test some other components than Dlib to make things run even faster.

      IPCam
    • CCTV on Openluup

      CatmanV2

      Can we? Simply? Specifically Foscams which used to run 'fine' on Vera but are not exactly high importance to me.

      But since they are there....

      Cheers

      C

      IPCam
    • Facial recognition explained and Customized Home-Assistant components

      rafale77

      Sharing what I have learned and some modifications to components with their benefits.
      On home assistant/python3, facial recognition involves the following steps:

      Establishing and maintaining a camera stream (over rtsp or http protocol) Have the ability to extract a single frame from the open stream in order to process it Pre process using the same steps as a video frame and store in memory a predetermined number of pictures as the known people to later compare with. In reality what is being compared are arrays of numbers generated by a model. Run a face detection and localization on the frame using one model Using the resulting location of 4., Extract from the picture, the face and encode it into a array of number Run a classification or comparison between the pre-set faces and the face on the video and spit out the "inference" or "prediction" to determine if they are close enough to be the same person.

      Even though a few components have been created on home-assistant for many years to do this, I ran into challenges which forced me to improve/optimize the process.

      Home Assistant's camera does not establish and keep open a stream in the background. It can open one on demand through its UI but doesn't keep it open. This forces the facial camera component to have to re-establish a new stream to get a single frame every time it needs to process an image causing up to 2s of delays, unacceptable for my application. I therefore rewrote the ffmpeg camera component to use opencv and maintain a stream within a python thread and since I have a GPU, I decided to decode the video using my GPU to relieve the CPU. This also required playing with some subtleties to avoid uselessly decoding frames we won't process while still needing to remove them from the thread buffer. The frame extraction was pretty challenging using ffmpeg which is why I opted to use opencv instead, as it executes the frame synchonization and alignment from the byte stream for us. The pre-set pictures was not a problem and a part of every face component. I started with the dlib component which had two models for ease of use. It makes use of the dlib library and the "facial_recognition" wrapper which has a python3 API but the CNN model requires a GPU and while it works well for me, turned out not to be the best as explained in this article and also quite resource intensive:https://www.learnopencv.com/face-detection-opencv-dlib-and-deep-learning-c-python/
      So I opted to move to the opencv DNN algorithm instead. Home Assistant has an openCV component but it is a bit generic and I couldn't figure out how to make it work. In any case, it did not have the steps 5 and 6 I wanted. For the face encoding step, I struggled quite a bit as it is quite directly connected to what option I would chose for step 6. From my investigation, I came to this: https://www.pyimagesearch.com/2018/09/24/opencv-face-recognition/

      "*Use dlib’s embedding model (but not it’s k-NN for face recognition)

      In my experience using both OpenCV’s face recognition model along with dlib’s face recognition model, I’ve found that dlib’s face embeddings are more discriminative, especially for smaller datasets.

      Furthermore, I’ve found that dlib’s model is less dependent on:

      Preprocessing such as face alignment
      Using a more powerful machine learning model on top of extracted face embeddings
      If you take a look at my original face recognition tutorial, you’ll notice that we utilized a simple k-NN algorithm for face recognition (with a small modification to throw out nearest neighbor votes whose distance was above a threshold).

      The k-NN model worked extremely well, but as we know, more powerful machine learning models exist.

      To improve accuracy further, you may want to use dlib’s embedding model, and then instead of applying k-NN, follow Step #2 from today’s post and train a more powerful classifier on the face embeddings.*"

      The trouble from my research is that I can see some people have tried but I have not seen posted anywhere a solution to translating the location array output from the opencv dnn model into a dlib rect object format for dlib to encode. Well, I did just that...

      For now I am sticking with the simple euclidian distance calculation and a distance threshold to determine the face match as it has been quite accurate for me but the option of going for a much more complex classification algorithm is open... when I get to it.

      So in summary, the outcome is modifications to:
      A. the ffmpeg camera component to switch to opencv and enable background maintenance of a stream with one rewritten file:
      https://github.com/rafale77/home-assistant/blob/dev/homeassistant/components/ffmpeg/camera.py
      B. Changes to the dlib face recognition component to support the opencv face detection model:
      https://github.com/rafale77/home-assistant/blob/dev/homeassistant/components/dlib_face_identify/image_processing.py
      C. Modified face_recognition wrapper to do the same, enabling conversion between dlib and opencv

      face_recognition/api.py at master · rafale77/face_recognition face_recognition/api.py at master · rafale77/face_recognition

      The world's simplest facial recognition api for Python and the command line - face_recognition/api.py at master · rafale77/face_recognition

      D. And additions of the new model to the face_recognition library involving adding a couple of files:
      face_recognition_models/face_recognition_models at master · rafale77/face_recognition_models face_recognition_models/face_recognition_models at master · rafale77/face_recognition_models

      Trained models for the face_recognition python library - face_recognition_models/face_recognition_models at master · rafale77/face_recognition_models

      init.py
      face_recognition_models/face_recognition_models/models at master · rafale77/face_recognition_models face_recognition_models/face_recognition_models/models at master · rafale77/face_recognition_models

      Trained models for the face_recognition python library - face_recognition_models/face_recognition_models/models at master · rafale77/face_recognition_models

      Overall these changes significantly improved speed and decreased cpu and gpu utilization rate over any of the original dlib components.
      At the moment the CUDA use for this inference is broken on openCV using the latest CUDA so I have not even switched on the GPU for facial detection yet (it worked fine using the dlib cnn model) but a fix may already have been posted so I will recompile openCV shortly...

      Edit: Sure enough openCV is fixed. I am running the face detection on the GPU now.

      IPCam
    • opensource NVR

      DesT

      At the moment i'm using the Surveillance software in Synology but I'm limited to 6 cameras (2 included and I took a 4pack a while ago)

      But I have 8 cameras, so right now, 2 of them are not in the NVR!

      I checked back in time motioneye but this software is very slow and all my cameras feed was lagging...

      any other solution? 😉

      IPCam
    • License Plate recognition

      rafale77

      Something fun to do if you have a camera located on your driveway:
      This home assistant component enables recognition of a license plate which in turn could open the garage door...

      Home Assistant Removed integration Removed integration

      The integration you requested has been removed

      IPCam
    • Monocle on Alexa

      rafale77

      Sharing an excellent skill I use to locally stream from my IPCams to echo shows:

      Monocle

      Your video stream does not need to go to the cloud. This skill just forwards the local stream address to the echo device when the camera name is called. It does require them to host the address and camera information (credentials) on their server though. I personally block all my IP cameras from accessing the internet from the router.

      IPCam
    For those who registered but didn't received the confirmation email, please send an email to support@smarthome.community with the email you used

    opensource NVR

    IPCam
    5
    23
    591
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • DesT
      DesT last edited by

      At the moment i'm using the Surveillance software in Synology but I'm limited to 6 cameras (2 included and I took a 4pack a while ago)

      But I have 8 cameras, so right now, 2 of them are not in the NVR!

      I checked back in time motioneye but this software is very slow and all my cameras feed was lagging...

      any other solution? 😉

      S W 2 Replies Last reply Reply Quote 0
      • rafale77
        rafale77 last edited by

        Nothing opensource. The best two that I know of are blue iris on windows and similar to what you are running... surveillance station and QVR on QNAP. My NAS setup happens to support a total of 20 cameras.

        1 Reply Last reply Reply Quote 0
        • DesT
          DesT last edited by DesT

          I'm playing with Shinobi since this morning...

          EDIT: Shinobi is having some plugin to play, like motion detection and object detection!

          1 Reply Last reply Reply Quote 0
          • rafale77
            rafale77 last edited by

            Shinobi was a video game from my youth... 🐷
            Nostalgia...

            CatmanV2 1 Reply Last reply Reply Quote 1
            • CatmanV2
              CatmanV2 @rafale77 last edited by

              @rafale77 said in opensource NVR:

              Shinobi was a video game from my youth... 🐷
              Nostalgia...

              Double Dragon, anyone 🙂

              I teach Kung Fu as well, so am biased 🙂

              Would love a decent, non-Windows CCTV solution given we're an all Mac location....

              C

              rafale77 1 Reply Last reply Reply Quote 0
              • rafale77
                rafale77 @CatmanV2 last edited by rafale77

                @CatmanV2 said in opensource NVR:

                Would love a decent, non-Windows CCTV solution given we're an all Mac location....

                C

                Depending on your requirements for recording, I would lean towards a NAS.
                For security, if you are running a separate LAN for your IP cameras a dedicated NVR may make sense.
                If not, then a NAS server on which you could run a few VMs would offer full CCTV control and even some automations... It gives you storage space, redundancy for the drives etc... QNAP moved from surveillance station to a new software called QVR, each supporting different licensing so... I ended up with one software supporting 8 cams and another supporting another 8 Cams, doubling the capacity but it means that I run two apps on my NAS doing the same thing. Not a big issue. It is that same NAS which hosts all my home automation stuff inside VMs as well!

                PS: yeah I loved double dragon too... 😉

                CatmanV2 1 Reply Last reply Reply Quote 0
                • S
                  sender @DesT last edited by sender

                  @DesT best advice: buy 2 more licences :-). I am running 13 cams succesfully on synology surveillance station. Around 65 to 70% cpu load with motion detection for 8 cams on synology and 3 on hikvision.
                  Tip: if playback is not smooth on the app, work on your wifi and prefer 5ghz insted of 2.4 WORLD OF DIFFERENCE!

                  1 Reply Last reply Reply Quote 0
                  • CatmanV2
                    CatmanV2 @rafale77 last edited by

                    @rafale77

                    Problem there is I'm throwing away a perfectly good NAS to implement a pretty clunky solution (VM is never going to be as elegant as native....)

                    C

                    rafale77 1 Reply Last reply Reply Quote 0
                    • rafale77
                      rafale77 @CatmanV2 last edited by rafale77

                      @CatmanV2
                      Yeah, not elegant I agree, but it seems like containers and VMs are all the rage these days. A number of my QNAP native Apps are now running as containers, including QVR pro which is the newer NVR software. Home Assistant is a prime example where "add ons" integrations are additional containers and it seems to be the direction the platform is going towards. It would in any case be a shame to throw away a good NAS away.
                      Maybe you can keep using it and just supplement it. That's the idea of building a cluster on proxmox to run and manage all my VMs and containers on a variety of hardware.

                      I tend to agree with @sender for @DesT, to keep it clean, I would just bite the bullet and buy 2 extra licenses if the NVR works for you, you may not find anything better to move all your cams to.

                      1 Reply Last reply Reply Quote 0
                      • CatmanV2
                        CatmanV2 last edited by

                        I guess there's the point that says 'When do I stop installing physical units and start VMing'

                        Can a Pi run Windows I wonder.

                        C

                        1 Reply Last reply Reply Quote 0
                        • rafale77
                          rafale77 last edited by rafale77

                          Yes but you need to have a minimum of physical units to share resources. I actually really dislike the container solutions and am finding it very clunky. Much more difficult to maintain and update than a VM. I understand the gain from not reserving resources and the fact that they are much more lightweight which enables them to multiply like rabbits living each in their own environments without interfering with one another but for my use case, the limited OS and environment just limit/defeat their purpose vs. a full VM or machine running its own lightweight OS. I think it is the right middle ground. Anyway getting way off topic and rambling a bit here as I am rethinking how to run my setup.

                          CatmanV2 1 Reply Last reply Reply Quote 1
                          • CatmanV2
                            CatmanV2 @rafale77 last edited by

                            This post is deleted!
                            1 Reply Last reply Reply Quote 0
                            • DesT
                              DesT last edited by

                              Looks like some fun are coming... a new server on his way with 2 GPU in it! Will try to play with Shinobi and object detection using GPU!

                              1 Reply Last reply Reply Quote 0
                              • rafale77
                                rafale77 last edited by rafale77

                                I too just ordered a mini pc with an nvidia GPU which I intend to use as a server... 😛

                                Will not be shinobi though...

                                1 Reply Last reply Reply Quote 0
                                • W
                                  wilme2 @DesT last edited by

                                  @DesT Is there a reason you wouldn't buy two more Synology licenses? The last time I bought one, I think I found them for $55 each...

                                  1 Reply Last reply Reply Quote 0
                                  • DesT
                                    DesT last edited by

                                    @wilme2 'cause I would like to have some "advanced" feature, like object/person detection and 'cause the Synology box we use for camera it's an old DS214 and it's not very fast and I would be more than happy to find something that I can put in a VM in our cluster to remove all other non-centralise "data" storage!

                                    I'm having a NAS storage of 100TB with an expansion already here that I can put another minimum of 100TB for future use.

                                    I already asked Synology to provide a way to install Surveillance in a VM so we can use it on our own hardware and of course they didn't want to go there!

                                    The exchange rate for USD->CAD is not also a good thing now. It's near 40% as today again!

                                    1 Reply Last reply Reply Quote 0
                                    • rafale77
                                      rafale77 last edited by

                                      I see... and I figure you don't want to upgrade your NAS?
                                      Wow you certainly have insane means on the hardware side of things.

                                      I have come to realize that my facial recognition setup is a bit slow so I am thinking about just supplementing it with a neural compute stick or an nvidia GPU which I can bridge into the VM of my NAS.

                                      1 Reply Last reply Reply Quote 0
                                      • DesT
                                        DesT last edited by

                                        @rafale77 We already have a 2 DELL Enterprise NAS (md3200i). Right now I'm using only the first enclosure of the first NAS for a total of 100TB, I'm planning to use the second enclosure later this year and put in 12x 10-12TB and after that I will be able to use the second NAS with 2 enclosure.

                                        I also have a SAS enclosure directly connected to our backup server too!

                                        So for sure I want to get ride of my 2 synology NAS.

                                        CatmanV2 1 Reply Last reply Reply Quote 0
                                        • CatmanV2
                                          CatmanV2 @DesT last edited by

                                          @DesT Can I just clarify: You run this at home?!

                                          C

                                          DesT 1 Reply Last reply Reply Quote 1
                                          • DesT
                                            DesT @CatmanV2 last edited by

                                            @CatmanV2 yeah! I convert a part of the house as a "datacenter"

                                            CatmanV2 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Welcome. If you’d like to participate in the discussion, rather than just read, then you can join the forum. As a member, you can interact with others here to share your experience and ask the questions you need answered.

                                            Powered by NodeBB | Contributors
                                            Hosted freely by PointPub Media Communications Inc. | Contact us