Reply
Member
Posts: 274
Registered: ‎03-01-2016
Kudos: 136

Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)

[ Edited ]

The achilles heel of Protect seems to be the hard drive in the CK2+.

 

The hard drive is the limiting factor in two ways:

  1. It will become the bottleneck before the processor does.
  2. It is limited to about 1,250 hours of storage.

 

The CK2+ Hard Drive is a Bottleneck

While the CK2+ is listed as supporting up to 20 cameras, I have found in the field that at about 12~15 cameras the performance of the viewers (both web and IOS app) becomes laggy. During playback video may halt, or be slow to start up. This happens even with excellent, direct LAN connections.

 

Some have suggested that using various non-default settings would help this. Of course, if you don't need the video quality that is the default you can compromise quality to get a more responsive experience. But in my fiddling around, the benefit of lowering frame rates (even down to 5 FPS) have been pretty minimal.

 

While experiencing laggy jerkey playback I have logged into the CK2+ and have looked at the performance meters. Memory is maxed out (97%) but processor utilization is very low, usually about 20%.

 

I have physically disconnected cameras from the network and responsiveness of Protect improves.

 

Unfortunately, there is no performance guage for the one thing that seems to most need one: Hard Drive occupancy.

 

Conclusion: The limiting factor on Protect running on a CK2+ is the hard drive performance.

 

 

The 1,250 hour storage limitation

Protect running on a stock CK2+ seems to have about 1,250 hours of storage. Admittedly this number is a little "squishy" but it is a better number to use for planning purposes than using the ostretch method of putting your head in a hole in the ground. It's a reasonable starting point.

 

I have not found that adjusting video frame rates (even down to 5 FPS) makes a very significant impact on the total hours of storage. The additional storage time that I observed by using slower FPS was quite modest, on the other of a 50% increase in hours stored. This is counter-intuitive, but has been my observation. There is some impact, but certainly nowhere near the 3x that you might expect.

 

My system utilizes 15 Pro G3 series cameras and I consider it to be maxed out. It is slow and sluggish, sometimes unresponsive when watching video playback. But the hours of rescordings was the greatest limitation in practical use.

 

The 1,250 hours breaks down roughly like this:

  • 5 Cameras: 10 days of retention.
  • 10 Cameras: 5 days of retention.
  • 15 Cameras: 3.5 days of retention.

This is quite inadequate for our needs, as it has turned out. The primary use we have for the cameras is legal defense against slips and falls and fraudulent accidents. For this you typically want longer storage of a minimum of 30 days and sometimes 3~6 months.

 

For a 15 camera system this would require something like 20TB of storage for just 1 month, and 100 TB for what we would really like.

 

The cost of purchasing this much storage isn't really an issue. A 72 TB RAID enterprise-class system costs roughly $1,500, or just about 1/3 the cost of the cameras and CK2+. That is, disk storage expense isn't a problem. Plus, these larger RAID systems tend to have very high performance. However, there is no provision for utilizing any form of external storage on Protect.

 

Some have suggested upgrading the 2.5" internal HDD to 5 TB Seagate drive. Aside from the whole issue with this class of hard drive utilizing overlapped tracks and having severely deteriorating performance over time, this just doesn't really fix the lack of storage. Having 7 days of storage instead of 3.5 doesn't really even move the needle for litigation defense archiving.

 

There is no ability to utilize external drives whether NAS, or USB, or SATA.

 

 

Conclusion:

The achilles heel of the Protect CK2+ system seems to be the hard disk.

  • The hard disk contains a minimal amount of storage (1,250 combined hours) which for many applications will be inadequate.
  • The hard disk will cause slow, even sluggish video playback as more cameras are added, and we have found 15 cameras to be the limit.

The remainder of the Protect system and Cameras seem just fine. The system works well and has an acceptable level fo features and functionality. Processor utilization is acceptable (at about 20%) and while memory utilization is high (at 97%) I am told tha much of this is simply caching and that high memory use isn't causing the Protect system to be thrashing the drive.

 

I believe that support for external USB or NAS storage is urgently needed if Protect is going to be a viable product in the loss prevention segment of video security.

 

 

-----

I know that UBNT and its supporters are quick to chime in with "But the CK2+ is just the first of a long and illustrious line of platforms for protect." While that may be the case, it is speculative and doesn't address the more pragmatic question of whether Protect running on a CK2+ is even a viable solution today, when more than 10~15 cameras OR more than 3 days of video storage is needed to be retained.

 

Ubiquiti Employee
Posts: 3,101
Registered: ‎02-13-2017
Kudos: 2682
Solutions: 174

Re: Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)

@SeattleDavid  - You are correct that the CK-G2+ is the entry point into the Protect system. We will have additional hardware available in the future. At $199, supporting 20 cameras already puts the CK-G2+ significantly ahead of comparable NVRs that only support 4 to 8 channels in this price range, and the zero monthly fees makes the long-term costs significantly less than using any of the cloud services.

 

We test all of our internal systems with 20 cameras (no UniFi Network running) and we have many customers running 15-20 cameras on the CK-G2+ with great performance. If you're seeing different results with UniFi Network disabled, we need to have you work with our customer support to get access to your system.

 

If you do have UniFi Network enabled, keep in mind that UniFi Network will consume some of the CK-G2+ resources, requiring a lower camera count. In the datasheet, we specify the CK-G2+ as supporting up to 15 cameras with UniFi Network enabled.

Member
Posts: 274
Registered: ‎03-01-2016
Kudos: 136

Re: Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)

I don’t know that the somewhat defensive “sales pitch” response helped the conversation. I am not questioning that there may be many markets for Protect.

 

My post was intended to be informational and to help prospective customers make an informed, intelligent, fact based decision. Then, if Protect and CK2+ is the right answer they can proceed in confidence.

 

UBNT is sometimes less than forthright or helpful in understanding exactly what the expectations from Protect should be. I realize that the YMMV argument can be used against any benchmark, but it shouldn’t be used to obfuscate reasonable facts. 

 

There is every evidence that:

 

1) Protect maxes out at 20 cameras and perhaps even becomes unresponsive and sluggish with 15.

 

2) The total storage is about 1,250 total hours.

 

3) The disk drive is most likely to be the bottleneck, not the processor. This is especially important to understand since disk drive occupancy does not appear as one of the gauges on the performance panel.

 

 

My posting was hopefully useful to people who wanted some idea of the total hours of video storage and of performance.

 

UBNT doesn’t want to sell its systems where they will underperform and so postings such as mine should be seen as a positive thing.

 

Sales pitchy responses to genuine and correct reviews is not a service to anybody. I like UBNT products, but they should be forthrightly described and positioned.

 

Ubiquiti Employee
Posts: 3,101
Registered: ‎02-13-2017
Kudos: 2682
Solutions: 174

Re: Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)

@SeattleDavid  If you're having performance problems we can arrange a customer support sessions to work on your system.

 

UniFi Protect on CK-G2+ is not expected to be sluggish with 20 cameras (Protect-only mode, UniFi Network disabled). If you are seeing sluggish performance, we need to have you work with customer support to understand why your system is not performing as expected.

 

I recorded this video of one of our 20-camera CK-G2+ in the office. It's Saturday here, so I'm accessing this system remotely over the internet, from across town. This system has 20 cameras, all UVC-G3-FLEX with standard settings. We have hundreds of G3-series cameras on this test rack connected to multiple CK-G2+ systems for performance testing.

 

This is the typical performance of UniFi Protect with 20 cameras:

 

 

If your system is not behaving like the video above, we will arrange for customer support to work with you to diagnose your specific performance problems. We do need to keep this forum on topic to your specific customer support problem, though.

Emerging Member
Posts: 78
Registered: ‎02-27-2018
Kudos: 26

Re: Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)

Wouldn't a solution be to enable the use of external storage with the UCK-G2-PLUS? Either via a network share, or there's that other USB-C port that's currently unused. I remember some discussion when it first came out as to whether that might be enabled at some point in future for external data use, but don't know what happened with that discussion.

Perhaps Ubnt could comment on whether either of these options are something that might happen in future?

Member
Posts: 117
Registered: ‎09-23-2016
Kudos: 36
Solutions: 1

Re: Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)

[ Edited ]

I’m probably one of the last to defend ubnt (heck, they’ve moderated some of my input on the video system) and while I agree that the system at the moment is consumer-level at best, my personal experience with the performance doesn’t mirror yours. 

 

I run 17 cameras, UniFi sdn controller, upgraded hdd to the smr 5 tb, and have zero performance issues. Nothing sluggish or slow. And storage is about 2 1/2 weeks fwiw. 

Member
Posts: 274
Registered: ‎03-01-2016
Kudos: 136

Re: Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)

This is interesting because it is dramatically different than my experience.

 

Are the cameras G3 (higher resolution) or older low resolution models? Are you using factory defaults for the camera settings?

 

I don’t understand how you can get 18 days of video with 17 cameras, despite the fact that the drive you are using is only 2.5x larger than stock.

 

Any suggestions of what may be different?

 

 

 

Member
Posts: 117
Registered: ‎09-23-2016
Kudos: 36
Solutions: 1

Re: Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)

I run default config. Cameras are G3. I just checked, and as of the time of this post, 2/4, recordings go back to early Jan 17th, so 18 days?

Established Member
Posts: 1,504
Registered: ‎01-29-2014
Kudos: 446
Solutions: 79

Re: Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)

The amount of storage used depends entirely on the activity on the camera. If you're recording an area which is mainly empty then the data generated between keyframes will be minimal.

 

I have two cameras and the (1TB) disk is 70% full after 10 weeks. There's not a lot of activity on either camera so inter-keyframe data generated is minimal.

Member
Posts: 117
Registered: ‎09-23-2016
Kudos: 36
Solutions: 1

Re: Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)


@Vestas wrote:

The amount of storage used depends entirely on the activity on the camera. If you're recording an area which is mainly empty then the data generated between keyframes will be minimal.


That's entirely true. Out of the 17 cameras that I have on my home setup, about 4-5 get about constant motion due to palm tree fronds being in close view and waving around with the slightest wind. The rest are mostly covering low-motion areas.

Member
Posts: 274
Registered: ‎03-01-2016
Kudos: 136

Re: Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)

This is still an oddity to me.

 

I get no no more than 3½ days, total. It’s a uselessly small amount of storage. I am using the stock 2TB drive.

 

Of the 15 cameras I have, three of them look into unlit black boxes (inside of a mailbox.) Four of them look into un-manned equipment rooms that have no moving equipment or blinking lights. Six of the cameras look at territorial views that are pretty static in my opinion. No people walking are und, etc. The last three view a driveway and road and have some activity.

 

I can’t imagine how you get over two weeks storage.

 

In any event, two weeks isn’t enough for most of my applications, in fraudulent slip and fall cases it’s necessary to have enough video to go back and look at a claim. Claimants will often wait a few weeks before notifying their victim that an accident occurred. So we would like to have a minimum of a month, and preferably three months of video. Video is still the best way of documenting what actually happened at an alleged accident scene.

 

Member
Posts: 117
Registered: ‎09-23-2016
Kudos: 36
Solutions: 1

Re: Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)


@SeattleDavid wrote:

In any event, two weeks isn’t enough for most of my applications, in fraudulent slip and fall cases it’s necessary to have enough video to go back and look at a claim. Claimants will often wait a few weeks before notifying their victim that an accident occurred. So we would like to have a minimum of a month, and preferably three months of video. Video is still the best way of documenting what actually happened at an alleged accident scene.

 


I hear ya. In an age of cheap storage, the inability to use a NAS (or even direct attached storage via USB) to store videos (or, at the very least, a means to batch offload or backup videos) is what renders Protect squarely a consumer product, and unsuitable for any true professional use.

Ubiquiti Employee
Posts: 4,423
Registered: ‎06-18-2015
Kudos: 1328
Solutions: 400

Re: Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)

@SeattleDavid,

The stock drive is 1TB, a 5X difference. To clarify about activity and required bitrate: the actual bitrate is based on scene complexity, it does not have anything to do with events or motion sensitivity. If, for example, those cameras in the mailbox have IR disabled and are recording a completely black scene, it will generate high bitrate due to the complexity of the image noise. Wind causing trees / plants / etc to sway will also generate a lot of movement (even if no events) and increase the complexity of the encoding thus resulting in a higher bitrate.

UBNT_Alternate_Logo.png
Ubiquiti Networks Enterprise Support Team


UniFi Protect: UniFi Protect Help Center | Frequently Asked Questions

UniFi Video 3: UniFi Video Help Center | UFV3 User Guide


Ubiquiti Employee
Posts: 4,423
Registered: ‎06-18-2015
Kudos: 1328
Solutions: 400

Re: Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)

@SeattleDavid,

When you have a moment, could you please generate a support file from within Protect and send me a public download link from a file share service like Dropbox via PM?

UBNT_Alternate_Logo.png
Ubiquiti Networks Enterprise Support Team


UniFi Protect: UniFi Protect Help Center | Frequently Asked Questions

UniFi Video 3: UniFi Video Help Center | UFV3 User Guide


Regular Member
Posts: 391
Registered: ‎04-15-2014
Kudos: 19
Solutions: 8

Re: Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)


@ncsakany wrote:

I’m probably one of the last to defend ubnt (heck, they’ve moderated some of my input on the video system) and while I agree that the system at the moment is consumer-level at best, my personal experience with the performance doesn’t mirror yours. 

 

I run 17 cameras, UniFi sdn controller, upgraded hdd to the smr 5 tb, and have zero performance issues. Nothing sluggish or slow. And storage is about 2 1/2 weeks fwiw. 


May I ask which FPS and resolution did you pick to get these figures ?

Member
Posts: 274
Registered: ‎03-01-2016
Kudos: 136

Re: Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)

Everything is factory default, except for three cameras which view inside a mailbox, and those are set at 5 FPS.

 

The system consistently has only 3 days of video using the stock drive, which is 2TB.

 

UBNT doesn’t provide any gauges or idiot lights or really anything to help know what is going on under the hood. For example, how much storage is allocated to each camera, or what the disk drive occupancy is. It’s impossible to manage if you believe in the philosophy “That which is measured can be managed.”

 

There are two issues here: 1) Retention period and 2) Laggyness. I consider the total storage (retention period) to be the more serious.

 

Even if several weeks of storage was possible by upgrading the disk to the absolute largest size possible, this would not be adequate for most litigation defense purposes (eg, documenting areas vulnerable to fraudulent slip and fall claims.)

 

Member
Posts: 117
Registered: ‎09-23-2016
Kudos: 36
Solutions: 1

Re: Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)


@oliv wrote:

@ncsakany wrote:

I’m probably one of the last to defend ubnt (heck, they’ve moderated some of my input on the video system) and while I agree that the system at the moment is consumer-level at best, my personal experience with the performance doesn’t mirror yours. 

 

I run 17 cameras, UniFi sdn controller, upgraded hdd to the smr 5 tb, and have zero performance issues. Nothing sluggish or slow. And storage is about 2 1/2 weeks fwiw. 


May I ask which FPS and resolution did you pick to get these figures ?


Default everything

Regular Member
Posts: 391
Registered: ‎04-15-2014
Kudos: 19
Solutions: 8

Re: Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)

A stupid idea:

What if cameras were "attached" to several different CKG2+ units ?

Teaching users which CKG2+ to use for each camera seems acceptable to me and you can solve both capacity and lagginess issues.

 

Ancient Member
Posts: 32,661
Registered: ‎05-05-2012
Kudos: 10449
Solutions: 1619

Re: Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)


@SeattleDavid wrote:

For example, how much storage is allocated to each camera...


There is no per camera allocation.  The only way to achieve that is to allocate fewer cameras per NVR.

Member
Posts: 274
Registered: ‎03-01-2016
Kudos: 136

Re: Protect: Understanding the Protect CK2+ Performance Limitations (The HDD is the achilles heel)

 I was not suggesting any sort of allocation. 

 

 I was suggesting information. Statistics. Data, facts.  I was suggesting some way of knowing which cameras were consuming the majority of the storage. 

 

 As I said in my posting: “that which is measured can be managed. “

 

 The CK 2+ provides a number of metrics in its control panel. Unfortunately, none of the metrics particularly help you in managing a protective system. 

Reply