How you can build practical applications by quantifying observations from video

Key Point Detection

https://us.norton.com/internetsecurity-iot-how-facial-recognition-software-works.html

Quantifying observations from video

  • Generating body key-point data from video (Open Pose)
  • Reading the key point data into Python (Pandas, JSON)
  • Performing analysis of the key-point data (Numpy, Pandas)
  • Creating custom video overlays or visualizations (Open CV)
  • Saving the original video with your newly generated overlays (Open CV)
https://github.com/CMU-Perceptual-Computing-Lab/openpose

Step 1 — Generating Body Key Point datasets using Open Pose

ffmpeg -i your_input_video.mov -vcodec h264 -acodec mp2 your_output_video.mp4
bin\OpenPoseDemo.exe — video examples\media\name_of_your_vid.mp4 — net_resolution “320x320” — display 0 — render_pose 0 — write_video output/name_of_output.avi
bin\OpenPoseDemo.exe --video examples\media\name_of_your_vid.mp4 --net_resolution "320x320" --part_candidates --write_json output/ --display 0 --render_pose 0

Step 2 — Reading in key point data into Python

// Result for BODY_25 (25 body parts consisting of COCO + foot)
// const std::map<unsigned int, std::string> POSE_BODY_25_BODY_PARTS {
// {0, "Nose"},
// {1, "Neck"},
// {2, "RShoulder"},
// {3, "RElbow"},
// {4, "RWrist"},
// {5, "LShoulder"},
// {6, "LElbow"},
// {7, "LWrist"},
// {8, "MidHip"},
// {9, "RHip"},
// {10, "RKnee"},
// {11, "RAnkle"},
// {12, "LHip"},
// {13, "LKnee"},
// {14, "LAnkle"},
// {15, "REye"},
// {16, "LEye"},
// {17, "REar"},
// {18, "LEar"},
// {19, "LBigToe"},
// {20, "LSmallToe"},
// {21, "LHeel"},
// {22, "RBigToe"},
// {23, "RSmallToe"},
// {24, "RHeel"},
// {25, "Background"}
// };
{
"version":1.1,
"people":[
{
"pose_keypoints_2d":[582.349,507.866,0.845918,746.975,631.307,0.587007,...],
"face_keypoints_2d":[468.725,715.636,0.189116,554.963,652.863,0.665039,...],
"hand_left_keypoints_2d":[746.975,631.307,0.587007,615.659,617.567,0.377899,...],
"hand_right_keypoints_2d":[617.581,472.65,0.797508,0,0,0,723.431,462.783,0.88765,...]
"pose_keypoints_3d":[582.349,507.866,507.866,0.845918,507.866,746.975,631.307,0.587007,...],
"face_keypoints_3d":[468.725,715.636,715.636,0.189116,715.636,554.963,652.863,0.665039,...],
"hand_left_keypoints_3d":[746.975,631.307,631.307,0.587007,631.307,615.659,617.567,0.377899,...],
"hand_right_keypoints_3d":[617.581,472.65,472.65,0.797508,472.65,0,0,0,723.431,462.783,0.88765,...]
}
],
// If `--part_candidates` enabled
"part_candidates":[
{
"0":[296.994,258.976,0.845918,238.996,365.027,0.189116],
"1":[381.024,321.984,0.587007],
"2":[313.996,314.97,0.377899],
"3":[238.996,365.027,0.189116],
"4":[283.015,332.986,0.665039],
"5":[457.987,324.003,0.430488,283.015,332.986,0.665039],
"6":[],
"7":[],
"8":[],
"9":[],
"10":[],
"11":[],
"12":[],
"13":[],
"14":[293.001,242.991,0.674305],
"15":[314.978,241,0.797508],
"16":[],
"17":[369.007,235.964,0.88765]
}
]
}

Step 3 — Performing analysis on the key point data

  • Distance between the knees
  • Rep count
  • Angle of the thigh relative to shin
  • Position of the knee relative to foot
  • Instantaneous velocity
  • Average velocity on ascent and decent

Step 4 — Creating custom video overlays in Open CV

cv2.imshow(‘video title’, frame)

Step 5 — Saving your video with overlays

out = cv2.VideoWriter(‘your_output_video.avi’,cv2.VideoWriter_fourcc(‘M’,’J’,’P’,’G’), 25, (width,height))
out.write(frame)

The path forward

Keen on practical machine learning?

Data Science, Product Strategy, Machine Learning

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Advanced Regular Expression Concepts in Python

Text Analysis in Healthcare

How Can We Analyze Time Series Data in the Era of COVID-19?

Formatting Strings

Deploying Data Dashboards Automatically, Reliably, and Securely

Outcome 2 — Part 3 — Challenges for Big Data in Healthcare

Challenges for Big Data in Healthcare

Automated Sample Storage Systems Market | Competitive Intelligence and Tracking Report 2022–2029

Behind The Curtain

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Sandy D

Sandy D

Data Science, Product Strategy, Machine Learning

More from Medium

Neural Style Transfer Guide

Automatic Evidence Validation with Machine Learning

1-Convex Sets and Functions (Modern Optimization Techniques)

ML Model Optimization for low compute environments