- Patent Number:
10922,577
- Appl. No:
16/666390
- Application Filed:
October 28, 2019
- نبذة مختصرة :
A technique and system for counting the number of repetitions of approximately the same action in an input video sequence using 3D convolutional neural networks is disclosed. The proposed system runs online and not on the complete video. It analyzes sequentially blocks of 20 non-consecutive frames. The cycle length within each block is evaluated using a deep network architecture and the information is then integrated over time. A unique property of the disclosed method is that it is shown to successfully train on entirely synthetic data, created by synthesizing moving random patches. It therefore effectively exploits the high generalization capability of deep neural networks. Coupled with a region of interest detection mechanism and a suitable mechanism to identify the time scale of the video, the system is robust enough to handle real world videos collected from youtube and elsewhere, as well as non-video signals such as sensor data revealing repetitious physical movement.
- Inventors:
Wolf, Lior (Herzlia, IL); Levy, Ofir (Jerusalem, IL)
- Claim:
1. A method of detecting and counting repetitive activity over time, comprising: receiving signal information from which repetitive activity is to be detected and counted; detecting repetitive activity from a plurality of time windows, where the time windows contain said signal information; utilizing a trained deep neural network classifier for estimating of a cycle length of each repetition of the repetitive activity within each time window; and integrating the estimation between a count-start and a count-stop, to produce a count of the number of the repetitions.
- Claim:
2. The method of claim 1 , further comprising selecting a window length for at least one time window based on uncertainty in estimating the cycle length of the repetitive activity.
- Claim:
3. The method of claim 1 , wherein said estimating of said cycle length comprises classifying said repetitive activity into a classification that is characterized by said cycle length.
- Claim:
4. The method of claim 1 , wherein said signal information comprises a video stream.
- Claim:
5. The method of claim 1 , wherein determining said count-start and said count-stop is based on an entropy of probabilities assigned by said deep neural network classifier.
- Claim:
6. A system for detecting and counting repetitive activity over time, comprising: a signal input to transfer, from outside to inside the system, signal information from which repetitive activity is detected and counted; one or more detectors for detecting repetitive activity from a plurality of time windows, where the time windows contain said signal information; a processor that utilizes a trained deep neural network classifier for determination of and outputting an estimation of a cycle length of each repetition of the repetitive activity within each time window; and at least two registers to integrate the estimation provided by the processor between a count-start and a count-stop and to produce a count of the number of the repetitions.
- Claim:
7. The system of claim 6 , wherein uncertainty in estimating the cycle length of the repetitive activity is used to select a window length for at least one time window.
- Claim:
8. The system of claim 6 , wherein the processor learns from a plurality of training examples.
- Claim:
9. The system of claim 8 , wherein the plurality of training examples includes synthetic unrealistic data.
- Claim:
10. The system of claim 6 , wherein said processor determines said estimation of said cycle length by classifying said repetitive activity into a classification that is characterized by said cycle length.
- Claim:
11. The system of claim 6 , wherein said signal information comprises a video stream.
- Claim:
12. The system of claim 6 , wherein said processor is configured for determining said count-start and count-stop based on an entropy of probabilities assigned by said deep neural network classifier.
- Claim:
13. A system for discerning whether repetitive activity is occurring over time, comprising: a signal input to transfer, from outside to inside the system, signal information from which it may be discerned whether repetitive activity has occurred; one or more detectors for detecting repetitive activity from a plurality of time windows, where the windows contain said information; and a processor that utilizes a trained deep neural network classifier for determination of and outputting whether a repetitive activity is occurring within each time window.
- Claim:
14. The system of claim 13 , wherein the detection of whether repetitive activity is occurring is based on uncertainty in estimating a cycle length as determined by the processor.
- Claim:
15. The system of claim 13 , wherein said processor outputs an estimation of a cycle length of said repetitive activity.
- Claim:
16. The system of claim 15 , wherein said processor determines said estimation of said cycle length by classifying said repetitive activity into a classification that is characterized by said cycle length.
- Claim:
17. The system of claim 13 , wherein said signal information comprises a video stream.
- Claim:
18. The system of claim 17 , wherein said processor produces said outputs online while said video stream is still incomplete.
- Claim:
19. The system of claim 17 , wherein said processor automatically detects, within each frame of said video stream, a region-of-interest containing said repetitive activity.
- Claim:
20. The system of claim 13 , wherein said processor is configured for determining whether a repetitive activity is occurring within each time window based on an entropy of probabilities assigned by said deep neural network classifier.
- Patent References Cited:
6795567 September 2004 Cham
7035435 April 2006 Li
7526101 April 2009 Avidan
7675655 March 2010 Marshall
8619187 December 2013 Garg
9183466 November 2015 Siskind
9269158 February 2016 Ukil
2003/0018475 January 2003 Basu
2004/0258307 December 2004 Viola
2006/0018516 January 2006 Masoud
2006/0112028 May 2006 Xiao
2009/0175496 July 2009 Kondo
2012/0008819 January 2012 Ding
2012/0008836 January 2012 Bobbitt
2012/0128212 May 2012 Almbladh
2013/0016239 January 2013 Cho
2017/0017857 January 2017 Wolf et al.
102281385 December 2011
102281385 December 2011
102281385 March 2013
103761746 April 2014
106663325 May 2017
WO 2015/132665 September 2015
WO-2015132665 December 2015
WO-2019000412 January 2019
- Other References:
International Preliminary Report on Patentability for PCT/IB2015/000728, dated Sep. 22, 2016. cited by applicant
International Search Report and Written Opinion for PCT/IB2015/000728, dated Sep. 21, 2015. cited by applicant
Notice of Allowance for U.S. Appl. No. 15/124,047, dated Jun. 19, 2019. cited by applicant
Office Action for U.S. Appl. No. 15/124,047, dated Feb. 25, 2019. cited by applicant
Office Action for U.S. Appl. No. 15/124,047, dated Sep. 25, 2018. cited by applicant
- Primary Examiner:
Tsai, Tsung Yin
- Attorney, Agent or Firm:
Devlin Law Firm LLC
Lennon, James M.
- الرقم المعرف:
edspgr.10922577
Processing Request
No Comments.