Frigate: Enabling Clips and Advanced Object Detection (pt2)

by Danny
7 comments

READING TIME: 9 MINUTES

In this guide, we’ll be taking our Home Assistant Frigate install to the next level with advanced object detection. We will be playing around with Masks and Zones, configuring Frigate to detect specific objects (such as dogs or vehicles), and enabling the snapshots and clips feature to save locally or to a network share.

Let’s get started!

This is Part 2 of a 3 part guide.


Frigate Install Series:

In Part 1, I’ve shown you how to:

  • Install Frigate within Home Assistant
  • Install the Frigate Home Assistant integration via HACS
  • Add your first camera to Frigate
  • Create a Lovelace card to display your camera

In Part 2 (this guide), I’ll show you how to:

  • Add your first object detection
  • Configure detection zones
  • Create masks
  • Advanced object detection (such as vehicles, dogs, or bicycles)

In Part 3 (postponed), I’ll show you how to:

  • Create a basic notification when an object is detected
  • Create an actionable notification that shows a screenshot
  • In the meantime, you can use this Frigate Notification blueprint

Prerequisites:

Before following this guide, make sure you have followed Part 1: Setting up Frigate in Home Assistant first. We will be editing and adding to the existing configuration file we created in Part 1.


What cameras am I using?

As I mentioned in Part 1, the cameras you choose must support RTSP (real time streaming protocol). I’ve been very happy with Amcrest RTSP camers so far and highly recommend them.

For indoor cameras, I’m using the Amcrest ProHD IP2M-841B:

Sale
Amcrest ProHD 1080P WiFi Camera 2MP (1920TVL) Indoor Pan/Tilt Security Wireless IP Camera IP2M-841B (Black)
  • HIGH PERFORMANCE, REAL-TIME, TRUE HD – Experience magnificent full-HD 1080P video at 30fps with enhanced low light capability utilizing the Sony Exmor IMX323 image sensor and Ambarella S2LM chipset. Extra-wide 90° viewing angle as well as remote pan/tilt and digital zoom allow you to cover more ground and keep more of what matters to you safe. Works with Alexa through AmcrestCloud. 360° Pan & 90° Tilt.
  • SMARTER SECURITY – Receive motion alert notifications, review footage and engage in two-way communication via your smartphone with the Amcrest View app. Playback and record professionally on a PC using Amcrest Surveillance Pro for Windows and MAC (included) or Blue Iris Professional. Works with Amcrest Cloud remote video storage, MicroSd card, Amcrest NVRs, Synology and QNAP NAS, FTP, Pale Moon and Sea Monkey Browsers, IE, Safari, Firefox 49.0 and Chrome with Amcrest Web View Extension.
  • SECURE CLOUD VIDEO BACKUP – The optional Amcrest Cloud remote video storage subscription service allows you automatically store your videos off-site in a third-party location hosted and secured by Amazon AWS. This way if something happens to your local PC/NVR/SDcard/NAS, the footage will be safely recorded in a secure off-site location and accessible to you through a web-based Flash interface for PC (Windows & MAC) (Chrome/Firefox/Safari/Edge) and AmcrestCloud smartphone app.

For outdoor cameras, I’m using the Amcrest IP5M-T1179EW-28MM:

Amcrest 5MP UltraHD Outdoor Security IP Turret PoE Camera with Mic/Audio, 5-Megapixel, 98ft NightVision, 2.8mm Lens, IP67 Weatherproof, MicroSD Recording (256GB), White (IP5M-T1179EW-28MM)
  • STARLIGHT ILLUMINATION & MIC – Stunning low-light 5MP performance with Starlight 1/2.7” image sensor and 5MP resolution (2592x1944). Super-wide 103° viewing angle allows you to cover more ground. Power Over Ethernet (POE camera) for easy installation (POE Injector or Switch Required; Not Included, Sold Separately). Does Not Include a Power Supply. This is a FIXED IP camera and Does Not Pan/Tilt. Built-in mic for one-way audio.
  • SMARTER SECURITY & STORAGE OPTIONS – Receive motion alert notifications and review footage via your smartphone with the Amcrest View app. Playback and record professionally on a PC using the Amcrest Surveillance Pro for Windows and MAC (included) or third-party software such as Blue Iris. Works with Amcrest Cloud remote video storage, MicroSD Card (up to 256GB), Amcrest NVRs, Synology and QNAP NAS, FTP, Edge, Chrome, Mozilla Firefox & Safari.
  • DAY OR NIGHT, RAIN OR SHINE – Built to weather any storm and the darkest of nights, this 5-Megapixel Turret PoE IP Camera features a heavy-duty weatherproof IP67 housing as well as built-in IR LEDs and a Starlight technology image sensor to achieve industry leading low-light performance and night-vision up to 98 feet. Dual H.265/H.264 compression allows for maximum encryption with the latest technology as well as backwards compatibility with legacy platforms.

Understanding Masks and Zones

I won’t go into this section in too much detail, as the Frigate documentation does a great job explaining it in detail. I’m personally not using zones or masks yet. If you’d like to configure masks or zones, you can do so by clicking the Frigate NVR integration in your sidebar, clicking a camera, and then choosing Show Options or Hide Options.

Masks should be used to “ignore” certain detection zones from your cameras point of view. For example, if you have a busy sidewalk in the front of your house and don’t want to record snapshots of passerby’s.

Zones allow you to define an area of a frame and apply additional object type filters so you can determine whether or not an object is detected within that zone. An example use case of zones might be detecting if a certain vehicle is detected in your driveway. Zones allow you to create some really automations, such as if a person object is detected at night, you could turn on your exterior lights or send you a notification.


Step 1: Adding more detection objects

This next section will allow you track specific objects in Frigate, such as dogs, vehicles, bicyles, and much more. A full list of available detection objects can be found here.

File Editor > frigate.yml. For this example, I want to detect person, dogs, and cell phones. Add the following code under your cameras: section

objects:
      track:
        - person
        - dog
        - cell phone

Then go Supervisor > Frigate NVR > Restart. No need to restart Home Assistant. You should always check the Log tab after every restart in case you have a syntax error. (If Frigate doesn’t load after making a change, you probably make a syntax or indentation error!)

Go back into Frigate NVR, click a camera, make sure “Bounding Boxes” is checked, and you should now see that is tracking PERSON, DOG, and CELL PHONE.

If your tracked objects only shows black thumbnails, that’s because it hasn’t yet tracked that object. After you let it detect a cellphone and a person, refresh the frigate page you’re on and it’ll show up correctly.

So far so good! You’ve now confirmed it tracks objects correctly. However, if you click the Events section, you’ll notice it’s not showing any events yet. This is because we haven’t configured it to capture screenshots or recording yet.


Step 2: Enable Frigate Snapshots

Go back to File Editor > frigate.yml.

In our current config, we have it set to just detect. There are 4 available roles: detect, clip, record, and rtmp. You can use all 4 if you’d like, or mix and match. For this example, I’m just adding clips which will save snapshots and recording clips.

  • detect: Detect and track objects
  • clip: Save motion clips (either to Home Assistant or any storage medium you choose)
    • See Step 88888 for saving to network share
  • record: Continuous recording
  • rtmp: Rebroadcast RTSP stream, instead of opening a new stream from Frigate

In your frigate.yml file, edit your config to match mine. I am adding the clips role, enabling snapshots, disabling the timestamp, and retaining clips for 3 days. You can also retain clips on a per object basis, so for example if you want to retain clips of persons for 7 days but dogs for 2, you can do that by reading over the Frigate documentation.

After saving, go back to Supervisor > Frigate NVR > Restart.

Snapshots will now be on by default:

Wait a few moments and then click Events. You show now see that Frigate has started capturing events, along with the timestamp, detection score, and the object label it detected.

If you click the thumbnail of the one of the images, you can get some more info if you’d like. You will notice that it says “No clip available”, which means there is no recording of the event yet, just a snapshot. This is because we haven’t turned on recording clips just yet.


Step 3: Enable Recording Clips

Head over to your frigate.yml file once again. Then, we’re going to add a few more lines of code. This time, we’re only going to retain clips for 1 day. Again, you can customize recording on a per-object basis if you’d like.

Save it once again, and then go back to Supervisor > Frigate NVR > Restart.

Frigate will show show that it’s recording clips:

Click the camera again. Now, walk in front of your camera or move a detected object in front of it. Wait a minute for the new events to process, and then click Events. This time when you click the thumbnail, it should show a video clip.

Awesome! So, now you can view snapshots and recording clips from Frigate NVR. That’s great, but I also wanted to let you know that you can view these from the Media Browser.

Step 4: View Snapshots and Recording from Media Browser

In your Home Assistant sidebar, click Media Browser.

You’ll see that a new Frigate folder has been created.

Remember, we only turned on “Clips” and not continuous recording, so your recording clips will be under Clips and nothing should be under Recordings.


Step 5: Review Home Assistant Frigate integration

Go to Configuration > Integrations. You’ll notice that Frigate probably hasn’t updated the number of new entities. There should be a bunch more now that you’ve got enabled snapshots and clips.

To update, click the 3 dots and then click Reload.

Click on 1 device, and you should see a few entities created, such as the cell phone, person, and dog.


Example Frigate.yml configuration

Here’s the code I’m using in my frigate.yml file. You are welcome to copy and paste this in, enter your own MQTT credentials and RTSP URL, and change up the detected objects and retain length.

mqtt:
  host: 192.168.68.124
  user: mqttuser
  password: password

cameras:
  patio_cam:
    ffmpeg:
      inputs:
        - path: rtsp://admin:password@192.168.68.99:554
          roles: 
            - detect
            - clips
    width: 1920
    height: 1080
    fps: 5
    objects:
      track:
        - person
        - dog
        - cell phone
    snapshots:
      enabled: True
      timestamp: false
      bounding_box: true
      retain:
        default: 3
    clips:
      enabled: True
      retain:
        default: 1
    
detectors: 
  cpu1:
    type: cpu
  cpu2:
    type: cpu

Wrapping Up

Now, we are one step closer to a fully automated local NVR setup within Home Assistant! The remaining step is to configure notifications which will be covered in Part 3.

Stay tuned for Part 3!


My Favorite Home Assistant Devices

Below are some of the Home Assistant-compatible devices I personally use in my home. I highly recommend each of them.

The full list of all Home Assistant compatible & recommended devices I use can be found on my Equipment List page.

7 comments

You may also like

7 comments

Eric August 4, 2021 - 7:50 am

Great tutorial. I’ve been using Frigate along with HA. It’s been more useful than Blue Iris, which I have running concurrently. I’ve been browsing through some of your other self hosted tutorial – super interesting! Thanks for all the work.

Reply
Danny September 1, 2021 - 4:21 am

I am so sorry for the late response. I replied back weeks ago but just now realized my comments weren’t actually sending. Anyway…

I’m glad you found my Frigate guide and some of my other posts helpful and interesting!

Reply
Heloish August 7, 2021 - 11:32 pm

Danny,
mate is there a way to store the files (clips and snapshots and recording) on a different external HDD mounted to the HA computer?
have searched the web, but there doesn’t seem to be any help available. Given we want to use this as a NVR it would be better to have the files stored on a external 2TB or greater hdd

Reply
peter dierckx September 1, 2021 - 8:07 am

hello danny

great additional manual to the existing material on the internet, not only this one but for all other subjects
Using frigate for a while, but extremely interested in how to store recordings on a network share. Looking forward for part three!!!!

Reply
Danny September 1, 2021 - 9:04 am

Hey Peter! Thank you, I enjoy documenting new projects as I go 🙂

Unfortunately, I haven’t been able to figure out how to store Frigate recordings on a network share. Frigate completely filled up my HA VM within just a couple days, so I ended up not sticking with Frigate for now and moved to Blue Iris on Windows, so I probably won’t get to a Part 3 for a while. I’m hoping Frigate pushes out an update in the UI that lets us point to a network share.

Reply
Babu September 4, 2021 - 9:25 pm

Is there any way to add schedule for recording the clips?. I want the camera to record on motion only during night. I am in the process of moving to Frigate from Blue Iris, this feature is important for me and couldn’t find a way to make it work.

Reply
Danny September 4, 2021 - 10:14 pm

I don’t know of a way to do that, but since Frigate is still pretty new, I’m sure it’ll be on the dev’s to-do list

Reply

Leave a Comment