reformat

backend/README.md

@@ -1,9 +1,17 @@
 ## pytorch-openpose
 
-pytorch implementation of [openpose](https://github.com/CMU-Perceptual-Computing-Lab/openpose) including **Body and Hand Pose Estimation**, and the pytorch model is directly converted from [openpose](https://github.com/CMU-Perceptual-Computing-Lab/openpose) caffemodel by [caffemodel2pytorch](https://github.com/vadimkantorov/caffemodel2pytorch). You could implement face keypoint detection in the same way if you are interested in. Pay attention to that the face keypoint detector was trained using the procedure described in [Simon et al. 2017] for hands.
-
-openpose detects hand by the result of body pose estimation, please refer to the code of [handDetector.cpp](https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/src/openpose/hand/handDetector.cpp).
+pytorch implementation of [openpose](https://github.com/CMU-Perceptual-Computing-Lab/openpose) including **Body and Hand
+Pose Estimation**, and the pytorch model is directly converted
+from [openpose](https://github.com/CMU-Perceptual-Computing-Lab/openpose) caffemodel
+by [caffemodel2pytorch](https://github.com/vadimkantorov/caffemodel2pytorch). You could implement face keypoint
+detection in the same way if you are interested in. Pay attention to that the face keypoint detector was trained using
+the procedure described in [Simon et al. 2017] for hands.
+
+openpose detects hand by the result of body pose estimation, please refer to the code
+of [handDetector.cpp](https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/src/openpose/hand/handDetector.cpp)
+.
 In the paper, it states as:
+
 ```
 This is an important detail: to use the keypoint detector in any practical situation, 
 we need a way to generate this bounding box. 
@@ -12,7 +20,9 @@ and use the wrist and elbow position to approximate the hand location,
 assuming the hand extends 0.15 times the length of the forearm in the same direction.
 ```
 
-If anybody wants a pure python wrapper, please refer to my [pytorch implementation](https://github.com/Hzzone/pytorch-openpose) of openpose, maybe it helps you to implement a standalone hand keypoint detector.
+If anybody wants a pure python wrapper, please refer to
+my [pytorch implementation](https://github.com/Hzzone/pytorch-openpose) of openpose, maybe it helps you to implement a
+standalone hand keypoint detector.
 
 Don't be mean to star this repo if it helps your research.
 
@@ -51,7 +61,7 @@ to run a demo with a feed from your webcam or run
 
     python demo.py
 
-to use a image from the images folder or run 
+to use a image from the images folder or run
 
     python demo_video.py <video-file>
 
@@ -60,6 +70,7 @@ to process a video file (requires [ffmpeg-python][ffmpeg]).
 [ffmpeg]: https://pypi.org/project/ffmpeg-python/
 
 ### Todo list
+
 - [x] convert caffemodel to pytorch.
 - [x] Body Pose Estimation.
 - [x] Hand Pose Estimation.
@@ -67,17 +78,21 @@ to process a video file (requires [ffmpeg-python][ffmpeg]).
 - [ ] Speed up.
 
 ### Demo
+
 #### Skeleton
 
 ![](images/skeleton.jpg)
+
 #### Body Pose Estimation
 
 ![](images/body_preview.jpg)
 
 #### Hand Pose Estimation
+
 ![](images/hand_preview.png)
 
 #### Body + Hand
+
 ![](images/demo_preview.png)
 
 #### Video Body
@@ -93,7 +108,9 @@ Attribution: [this video](https://www.youtube.com/watch?v=kc-e129SBb4).
 Attribution: [this video](https://www.youtube.com/watch?v=yOAmYSW3WyU).
 
 ### Citation
-Please cite these papers in your publications if it helps your research (the face keypoint detector was trained using the procedure described in [Simon et al. 2017] for hands):
+
+Please cite these papers in your publications if it helps your research (the face keypoint detector was trained using
+the procedure described in [Simon et al. 2017] for hands):
 
 ```
 @inproceedings{cao2017realtime,

backend/advice.py

@@ -1,80 +1,80 @@
 def ad_jump(result):
-    ret=""
-    if(result["jump_power"]["level"]==3):
-        ret=ret+"Need to do more to promote leg muscle strength. "
-    elif(result["jump_power"]["level"]==4):
-        ret="The leg muscle is weak and the position needs to be improved. "
+    ret = ""
+    if (result["jump_power"]["level"] == 3):
+        ret = ret + "Need to do more to promote leg muscle strength. "
+    elif (result["jump_power"]["level"] == 4):
+        ret = "The leg muscle is weak and the position needs to be improved. "
 
-    if(result["core_strength"]["level"]==3):
-        ret=ret+"The core strength need to be improved. "
-    elif result["core_strength"]["level"]==4:
-        ret=ret+"More practice to do to promote the core strength. "
+    if (result["core_strength"]["level"] == 3):
+        ret = ret + "The core strength need to be improved. "
+    elif result["core_strength"]["level"] == 4:
+        ret = ret + "More practice to do to promote the core strength. "
 
-    if(result["landing_position"]["level"]==3):
-        ret=ret+"The landing position is not standard enough. Take some practice on the leg. "
-    elif result["landing_position"]["level"]==4:
-        ret=ret+"The landing position is far from standard, see some relavant video to correct posture. "
-    if(ret==""):
-        ret="The jump is good and keep it!"
+    if (result["landing_position"]["level"] == 3):
+        ret = ret + "The landing position is not standard enough. Take some practice on the leg. "
+    elif result["landing_position"]["level"] == 4:
+        ret = ret + "The landing position is far from standard, see some relavant video to correct posture. "
+    if (ret == ""):
+        ret = "The jump is good and keep it!"
     return ret
 
 def ad_juanfu(result):
-    ret=""
-    if(result["head_power"]["level"]==3):
-        ret=ret+"Need to do more to promote head muscle strength. "
-    elif(result["head_power"]["level"]==4):
-        ret="The neck and abdominal muscles are weak and the position needs to be improved. "
+    ret = ""
+    if (result["head_power"]["level"] == 3):
+        ret = ret + "Need to do more to promote head muscle strength. "
+    elif (result["head_power"]["level"] == 4):
+        ret = "The neck and abdominal muscles are weak and the position needs to be improved. "
 
-    if(result["core_strength"]["level"]==3):
-        ret=ret+"The core strength need to be improved.See some relavent video to promote it. "
-    elif result["core_strength"]["level"]==4:
-        ret=ret+"More practice to do to promote the core strength. "
+    if (result["core_strength"]["level"] == 3):
+        ret = ret + "The core strength need to be improved.See some relavent video to promote it. "
+    elif result["core_strength"]["level"] == 4:
+        ret = ret + "More practice to do to promote the core strength. "
 
-    if(result["body_stability"]["level"]==3):
-        ret=ret+"The body_stability is not enough. Take some practice on the leg. "
-    elif result["body_stability"]["level"]==4:
-        ret=ret+"The body_stability is far from standard, see some relavant video to correct posture . "
-    if(ret==""):
-        ret="The abdominal campaign is good and keep it!"
+    if (result["body_stability"]["level"] == 3):
+        ret = ret + "The body_stability is not enough. Take some practice on the leg. "
+    elif result["body_stability"]["level"] == 4:
+        ret = ret + "The body_stability is far from standard, see some relavant video to correct posture . "
+    if (ret == ""):
+        ret = "The abdominal campaign is good and keep it!"
     return ret
 
 def ad_pingban(result):
-    ret=""
-    if(result["arm_strength"]["level"]==3):
-        ret=ret+"Need to do more to promote arm muscle strength. "
-    elif(result["arm_strength"]["level"]==4):
-        ret="The forearm strength are inadequate and the position needs to be improved. "
+    ret = ""
+    if (result["arm_strength"]["level"] == 3):
+        ret = ret + "Need to do more to promote arm muscle strength. "
+    elif (result["arm_strength"]["level"] == 4):
+        ret = "The forearm strength are inadequate and the position needs to be improved. "
 
-    if(result["stability"]["level"]==3):
-        ret=ret+"The core strength need to be improved.Try to lengthen duration time and frequency. "
-    elif result["stability"]["level"]==4:
-        ret=ret+"More practice to do to promote the stability . "
+    if (result["stability"]["level"] == 3):
+        ret = ret + "The core strength need to be improved.Try to lengthen duration time and frequency. "
+    elif result["stability"]["level"] == 4:
+        ret = ret + "More practice to do to promote the stability . "
 
-    if(result["standard"]["level"]==3):
-        ret=ret+"Plank movements are less than standard and require more straight leg support. "
-    elif result["standard"]["level"]==4:
-        ret=ret+"The body posture is far from standard ,see the standard video and improve leg strength. "
-    if(ret==""):
-        ret="Plank moves are fine, and your goal is to maximize your endurance!"
+    if (result["standard"]["level"] == 3):
+        ret = ret + "Plank movements are less than standard and require more straight leg support. "
+    elif result["standard"]["level"] == 4:
+        ret = ret + "The body posture is far from standard ,see the standard video and improve leg strength. "
+    if (ret == ""):
+        ret = "Plank moves are fine, and your goal is to maximize your endurance!"
     return ret
 
 def ad_shendun(result):
-    if(result["leg_strength"]["level"]==1):
+    if (result["leg_strength"]["level"] == 1):
         return "The posture is good and remember to enhance endurance. "
-    elif result["leg_strength"]["level"]==2:
+    elif result["leg_strength"]["level"] == 2:
         return "Try a wider squat."
 
-    elif result["leg_strength"]["level"]==3:
+    elif result["leg_strength"]["level"] == 3:
         return "Squat range is far from enough, more practice thigh strength. "
     else:
         return "Check out some video to learn about standard postures and work on your thigh and calf muscles"
 
 def ad_gaotaitui(result):
-    if(result["standard"]["level"]==1):
+    if (result["standard"]["level"] == 1):
         return "The posture is good and remember to enhance endurance. "
-    elif result["standard"]["level"]==2:
+    elif result["standard"]["level"] == 2:
         return "Try to elevate your thighs to increase intensity. "
-    elif result["standard"]["level"]==3:
+    elif result["standard"]["level"] == 3:
         return "High leg lift is not enough, more practice thigh strength. "
     else:
         return "Check out some video to learn about standard postures and work on your thigh and calf muscles"

backend/analyse_by_npy.py

@@ -1,8 +1,9 @@
-import numpy as np
 import cv2 as cv
 import math
+import numpy as np
 import sys
-from demo_func import run_openpose_for_image_front,run_openpose_for_image_side
+
+from demo_func import run_openpose_for_image_front, run_openpose_for_image_side
 
 sys.path.append('../')
 
@@ -40,7 +41,6 @@ HIP_OFFSET_ARGUMENT_0 = 0.5
 HIP_OFFSET_ARGUMENT_1 = 1
 HIP_OFFSET_ARGUMENT_2 = 3
 
-
 UP_FORWARD_ARGUMENT_0 = 2
 UP_FORWARD_ARGUMENT_1 = 10
 UP_FORWARD_ARGUMENT_2 = 20
@@ -53,50 +53,43 @@ KNEE_FORWARD_ARGUMENT_0 = 1
 KNEE_FORWARD_ARGUMENT_1 = 5
 KNEE_FORWARD_ARGUMENT_2 = 10
 
-
-
-
 source_npy_front = "pose_temp_data/{}-0.npy".format(index)
 source_npy_side = "pose_temp_data/{}-1.npy".format(index)
 save_path = "pose_processed_txt/{}.txt".format(index)
 
-def text_save(filename, data):#filename为写入CSV文件的路径，data为要写入数据列表.
-    file = open(filename,'w',encoding='utf-8')
+def text_save(filename, data):  # filename为写入CSV文件的路径，data为要写入数据列表.
+    file = open(filename, 'w', encoding='utf-8')
     for i in range(len(data)):
-        s = str(data[i]).replace('[','').replace(']','')
-        s = s.replace("'",'').replace(',','') +'\n'
+        s = str(data[i]).replace('[', '').replace(']', '')
+        s = s.replace("'", '').replace(',', '') + '\n'
         file.write(s)
     file.close()
     print("保存成功")
 
 def angle(v1, v2):
-  dx1 = v1[2] - v1[0]
-  dy1 = v1[3] - v1[1]
-  dx2 = v2[2] - v2[0]
-  dy2 = v2[3] - v2[1]
-  angle1 = math.atan2(dy1, dx1)
-  angle1 = round(angle1 * 180.0/math.pi,2)
-  # print(angle1)
-  angle2 = math.atan2(dy2, dx2)
-  angle2 = round(angle2 * 180.0/math.pi,2)
-  # print(angle2)
-  if angle1*angle2 >= 0:
-    included_angle = abs(angle1-angle2)
-  else:
-    included_angle = abs(angle1) + abs(angle2)
-    if included_angle > 180:
-      included_angle = 360 - included_angle
-  return included_angle
+    dx1 = v1[2] - v1[0]
+    dy1 = v1[3] - v1[1]
+    dx2 = v2[2] - v2[0]
+    dy2 = v2[3] - v2[1]
+    angle1 = math.atan2(dy1, dx1)
+    angle1 = round(angle1 * 180.0 / math.pi, 2)
+    # print(angle1)
+    angle2 = math.atan2(dy2, dx2)
+    angle2 = round(angle2 * 180.0 / math.pi, 2)
+    # print(angle2)
+    if angle1 * angle2 >= 0:
+        included_angle = abs(angle1 - angle2)
+    else:
+        included_angle = abs(angle1) + abs(angle2)
+        if included_angle > 180:
+            included_angle = 360 - included_angle
+    return included_angle
 
 def cal_head_extra_stress(angle):
-    return (HEAD_STRESS_ARGUMENT_1 * angle + HEAD_STRESS_ARGUMENT_2 * pow(angle,2) \
-           + HEAD_STRESS_ARGUMENT_3 * pow(angle,3) + HEAD_STRESS_ARGUMENT_4 * pow(angle,4))/HEAD_STRESS_ARGUMENT_0
-
-
+    return (HEAD_STRESS_ARGUMENT_1 * angle + HEAD_STRESS_ARGUMENT_2 * pow(angle, 2) \
+            + HEAD_STRESS_ARGUMENT_3 * pow(angle, 3) + HEAD_STRESS_ARGUMENT_4 * pow(angle, 4)) / HEAD_STRESS_ARGUMENT_0
 
 def analyse_npy_front(npy_front):
-
-
     result = []
     data = np.load(npy_front)
 
@@ -106,18 +99,15 @@ def analyse_npy_front(npy_front):
     print("正面分析内容如下：")
     result.append("正面分析内容如下：")
 
-
-
     # h-eye - h-elbow
-    h_pixel_eye2elbow = abs(data[14][1]-data[3][1])
-    #print("h_pixel{}".format(h_pixel_eye2elbow))
+    h_pixel_eye2elbow = abs(data[14][1] - data[3][1])
+    # print("h_pixel{}".format(h_pixel_eye2elbow))
 
     h_real_eye2elbow = float(height) * 0.333
-    #print("h_real{}".format(h_real_eye2elbow))
+    # print("h_real{}".format(h_real_eye2elbow))
 
     # body mid x
-    ankle_mid_x = (data[13][0] + data[10][0])/2
-
+    ankle_mid_x = (data[13][0] + data[10][0]) / 2
 
     # for head
     ear_r_x = data[16][0]
@@ -129,20 +119,19 @@ def analyse_npy_front(npy_front):
     ear_level = ""
     if head_angle <= level_0:
         ear_level = ""
-    elif head_angle <= level_1 and head_angle > level_0 :
+    elif head_angle <= level_1 and head_angle > level_0:
         ear_level = "轻微"
-    elif head_angle > level_1 and head_angle <= level_2 :
+    elif head_angle > level_1 and head_angle <= level_2:
         ear_level = "中等"
     else:
         ear_level = "严重"
 
-
     if ear_l_y < ear_r_y and ear_level != "":
-        print("头部分析：头部左部高，向右倾斜，角度为{}度，程度{}".format(head_angle,ear_level))
-        result.append("头部分析：头部左部高，向右倾斜，角度为{}度，程度{}".format(head_angle,ear_level))
+        print("头部分析：头部左部高，向右倾斜，角度为{}度，程度{}".format(head_angle, ear_level))
+        result.append("头部分析：头部左部高，向右倾斜，角度为{}度，程度{}".format(head_angle, ear_level))
     elif ear_l_y > ear_r_y and ear_level != "":
-        print("头部分析：头部右部高，向左倾斜，角度为{}度，程度{}".format(head_angle,ear_level))
-        result.append("头部分析：头部右部高，向左倾斜，角度为{}度，程度{}".format(head_angle,ear_level))
+        print("头部分析：头部右部高，向左倾斜，角度为{}度，程度{}".format(head_angle, ear_level))
+        result.append("头部分析：头部右部高，向左倾斜，角度为{}度，程度{}".format(head_angle, ear_level))
     else:
         print("头部分析：头部保持完美平衡")
         result.append("头部分析：头部保持完美平衡")
@@ -152,11 +141,11 @@ def analyse_npy_front(npy_front):
     shoulder_r_y = data[2][1]
     shoulder_l_x = data[5][0]
     shoulder_l_y = data[5][1]
-    shoulder_mid_x = (shoulder_l_x + shoulder_r_x)/2
-    offset_x_shoulder = (shoulder_mid_x - ankle_mid_x)*h_real_eye2elbow/h_pixel_eye2elbow
+    shoulder_mid_x = (shoulder_l_x + shoulder_r_x) / 2
+    offset_x_shoulder = (shoulder_mid_x - ankle_mid_x) * h_real_eye2elbow / h_pixel_eye2elbow
     d_shoulder = abs(data[5][1] - data[2][1])
     l_shoulder = math.sqrt((shoulder_l_x - shoulder_r_x) ** 2 + (shoulder_l_y - shoulder_r_y) ** 2)
-    rate_shoulder = d_shoulder/l_shoulder
+    rate_shoulder = d_shoulder / l_shoulder
     shoulder_angle = angle([shoulder_r_x, shoulder_r_y, shoulder_l_x, shoulder_l_y], [0, 0, 1, 0])
     shoulder_level = ""
     shoulder_offset_level = ""
@@ -164,7 +153,7 @@ def analyse_npy_front(npy_front):
         shoulder_level = ""
     elif shoulder_angle <= level_1 and shoulder_angle > level_0:
         shoulder_level = "轻微"
-    elif shoulder_angle > level_1 and shoulder_angle <= level_2 :
+    elif shoulder_angle > level_1 and shoulder_angle <= level_2:
         shoulder_level = "中等"
     else:
         shoulder_level = "严重"
@@ -173,29 +162,33 @@ def analyse_npy_front(npy_front):
         shoulder_offset_level = ""
     elif offset_x_shoulder <= SHOULDER_OFFSET_ARGUMENT_1 and offset_x_shoulder > SHOULDER_OFFSET_ARGUMENT_0:
         shoulder_offset_level = "轻微"
-    elif offset_x_shoulder > SHOULDER_OFFSET_ARGUMENT_1 and offset_x_shoulder <= SHOULDER_OFFSET_ARGUMENT_2 :
+    elif offset_x_shoulder > SHOULDER_OFFSET_ARGUMENT_1 and offset_x_shoulder <= SHOULDER_OFFSET_ARGUMENT_2:
         shoulder_offset_level = "中等"
     else:
         shoulder_offset_level = "严重"
 
     print("肩部分析：")
     result.append("肩部分析：")
-    if offset_x_shoulder > 0  and shoulder_offset_level != "":
-        print("肩部向左横移，距离为{}cm，程度{}".format(round(abs(offset_x_shoulder),3),shoulder_offset_level))
-        result.append("肩部向左横移，距离为{}cm，程度{}".format(round(abs(offset_x_shoulder),3),shoulder_offset_level))
+    if offset_x_shoulder > 0 and shoulder_offset_level != "":
+        print("肩部向左横移，距离为{}cm，程度{}".format(round(abs(offset_x_shoulder), 3), shoulder_offset_level))
+        result.append("肩部向左横移，距离为{}cm，程度{}".format(round(abs(offset_x_shoulder), 3), shoulder_offset_level))
     elif offset_x_shoulder < 0 and shoulder_offset_level != "":
-        print("肩部向右横移，距离为{}cm，程度{}".format(round(abs(offset_x_shoulder),3),shoulder_offset_level))
-        result.append("肩部向右横移，距离为{}cm，程度{}".format(round(abs(offset_x_shoulder),3),shoulder_offset_level))
+        print("肩部向右横移，距离为{}cm，程度{}".format(round(abs(offset_x_shoulder), 3), shoulder_offset_level))
+        result.append("肩部向右横移，距离为{}cm，程度{}".format(round(abs(offset_x_shoulder), 3), shoulder_offset_level))
     else:
         print("肩部中心位居中轴，位置良好")
         result.append("肩部中心位居中轴，位置良好")
 
     if shoulder_l_y < shoulder_r_y and shoulder_level != "":
-        print("肩部左部高，向右倾斜，角度为{}度，程度{}，高度差异与肩长比值为{}%".format(shoulder_angle,shoulder_level,round(rate_shoulder*100,2)))
-        result.append("肩部左部高，向右倾斜，角度为{}度，程度{}，高度差异与肩长比值为{}%".format(shoulder_angle,shoulder_level,round(rate_shoulder*100,2)))
+        print("肩部左部高，向右倾斜，角度为{}度，程度{}，高度差异与肩长比值为{}%".format(shoulder_angle, shoulder_level,
+                                                            round(rate_shoulder * 100, 2)))
+        result.append("肩部左部高，向右倾斜，角度为{}度，程度{}，高度差异与肩长比值为{}%".format(shoulder_angle, shoulder_level,
+                                                                    round(rate_shoulder * 100, 2)))
     elif ear_l_y > ear_r_y and shoulder_level != "":
-        print("肩部右部高，向左倾斜，角度为{}度，程度{}，高度差异与肩长比值为{}%".format(shoulder_angle,shoulder_level,round(rate_shoulder*100,2)))
-        result.append("肩部右部高，向左倾斜，角度为{}度，程度{}，高度差异与肩长比值为{}%".format(shoulder_angle,shoulder_level,round(rate_shoulder*100,2)))
+        print("肩部右部高，向左倾斜，角度为{}度，程度{}，高度差异与肩长比值为{}%".format(shoulder_angle, shoulder_level,
+                                                            round(rate_shoulder * 100, 2)))
+        result.append("肩部右部高，向左倾斜，角度为{}度，程度{}，高度差异与肩长比值为{}%".format(shoulder_angle, shoulder_level,
+                                                                    round(rate_shoulder * 100, 2)))
     else:
         print("肩部角度保持完美平衡")
         result.append("肩部角度保持完美平衡")
@@ -205,11 +198,11 @@ def analyse_npy_front(npy_front):
     hip_r_y = data[8][1]
     hip_l_x = data[11][0]
     hip_l_y = data[11][1]
-    hip_mid_x = (hip_r_x + hip_l_x)/2
-    offset_x_hip = (hip_mid_x - ankle_mid_x)*h_real_eye2elbow/h_pixel_eye2elbow
+    hip_mid_x = (hip_r_x + hip_l_x) / 2
+    offset_x_hip = (hip_mid_x - ankle_mid_x) * h_real_eye2elbow / h_pixel_eye2elbow
     hd_hip = abs(data[11][1] - data[8][1])
     d_hip = math.sqrt((hip_l_x - hip_r_x) ** 2 + (hip_l_y - hip_r_y) ** 2)
-    rate_hip = hd_hip/d_hip
+    rate_hip = hd_hip / d_hip
     hip_angle = angle([hip_r_x, hip_r_y, hip_l_x, hip_l_y], [0, 0, 1, 0])
     hip_level = ""
     hip_offset_level = ""
@@ -217,7 +210,7 @@ def analyse_npy_front(npy_front):
         hip_level = ""
     elif hip_angle <= level_1 and hip_angle > level_0:
         hip_level = "轻微"
-    elif hip_angle > level_1 and hip_angle <= level_2 :
+    elif hip_angle > level_1 and hip_angle <= level_2:
         hip_level = "中等"
     else:
         hip_level = "严重"
@@ -226,34 +219,33 @@ def analyse_npy_front(npy_front):
         hip_offset_level = ""
     elif offset_x_hip <= HIP_OFFSET_ARGUMENT_1 and offset_x_hip > HIP_OFFSET_ARGUMENT_0:
         hip_offset_level = "轻微"
-    elif offset_x_hip > HIP_OFFSET_ARGUMENT_1 and offset_x_hip <= HIP_OFFSET_ARGUMENT_2 :
+    elif offset_x_hip > HIP_OFFSET_ARGUMENT_1 and offset_x_hip <= HIP_OFFSET_ARGUMENT_2:
         hip_offset_level = "中等"
     else:
         hip_offset_level = "严重"
     print("髋部分析：")
     result.append("髋部分析：")
 
-    if offset_x_hip > 0  and hip_offset_level != "":
-        print("髋部向左横移，距离为{}cm，程度{}".format(round(abs(offset_x_hip),3),hip_offset_level))
-        result.append("髋部向左横移，距离为{}cm，程度{}".format(round(abs(offset_x_hip),3),hip_offset_level))
+    if offset_x_hip > 0 and hip_offset_level != "":
+        print("髋部向左横移，距离为{}cm，程度{}".format(round(abs(offset_x_hip), 3), hip_offset_level))
+        result.append("髋部向左横移，距离为{}cm，程度{}".format(round(abs(offset_x_hip), 3), hip_offset_level))
     elif offset_x_hip < 0 and hip_offset_level != "":
-        print("髋部向右横移，距离为{}cm，程度{}".format(round(abs(offset_x_hip),3),hip_offset_level))
-        result.append("髋部向右横移，距离为{}cm，程度{}".format(round(abs(offset_x_hip),3),hip_offset_level))
+        print("髋部向右横移，距离为{}cm，程度{}".format(round(abs(offset_x_hip), 3), hip_offset_level))
+        result.append("髋部向右横移，距离为{}cm，程度{}".format(round(abs(offset_x_hip), 3), hip_offset_level))
     else:
         print("髋部中心位居中轴，位置良好")
         result.append("髋部中心位居中轴，位置良好")
 
     if hip_l_y < hip_r_y:
-        print("髋部左部高，向右倾斜，角度为{}度，程度{}，高度差异与髋宽比值为{}%".format(hip_angle,hip_level,round(rate_hip*100,2)))
-        result.append("髋部左部高，向右倾斜，角度为{}度，程度{}，高度差异与髋宽比值为{}%".format(hip_angle,hip_level,round(rate_hip*100,2)))
+        print("髋部左部高，向右倾斜，角度为{}度，程度{}，高度差异与髋宽比值为{}%".format(hip_angle, hip_level, round(rate_hip * 100, 2)))
+        result.append("髋部左部高，向右倾斜，角度为{}度，程度{}，高度差异与髋宽比值为{}%".format(hip_angle, hip_level, round(rate_hip * 100, 2)))
     elif ear_l_y > ear_r_y:
-        print("髋部右部高，向左倾斜，角度为{}度，程度{}，高度差异与髋宽比值为{}%".format(hip_angle,hip_level,round(rate_hip*100,2)))
-        result.append("髋部右部高，向左倾斜，角度为{}度，程度{}，高度差异与髋宽比值为{}%".format(hip_angle,hip_level,round(rate_hip*100,2)))
+        print("髋部右部高，向左倾斜，角度为{}度，程度{}，高度差异与髋宽比值为{}%".format(hip_angle, hip_level, round(rate_hip * 100, 2)))
+        result.append("髋部右部高，向左倾斜，角度为{}度，程度{}，高度差异与髋宽比值为{}%".format(hip_angle, hip_level, round(rate_hip * 100, 2)))
     else:
         print("髋部角度保持完美平衡")
         result.append("髋部角度保持完美平衡")
 
-
     #
 
     print("腿型分析：")
@@ -266,10 +258,8 @@ def analyse_npy_front(npy_front):
     knee_l_y = data[12][1]
     d_knee = math.sqrt((knee_l_x - knee_r_x) ** 2 + (knee_l_y - knee_r_y) ** 2)
     rate_knee = d_knee / d_hip
-    print("双膝间距与髋宽比值为{}%".format(round(rate_knee*100,2)))
-    result.append("双膝间距与髋宽比值为{}%".format(round(rate_knee*100,2)))
-
-
+    print("双膝间距与髋宽比值为{}%".format(round(rate_knee * 100, 2)))
+    result.append("双膝间距与髋宽比值为{}%".format(round(rate_knee * 100, 2)))
 
     # XO judgement
     ankle_r_x = data[10][0]
@@ -278,13 +268,13 @@ def analyse_npy_front(npy_front):
     ankle_l_y = data[13][1]
     d_ankle = math.sqrt((ankle_l_x - ankle_r_x) ** 2 + (ankle_l_y - ankle_r_y) ** 2)
     rate_ankle = d_ankle / d_knee
-    print("脚踝间距与双膝间距比值为{}%".format(round(rate_ankle*100,2)))
-    result.append("脚踝间距与双膝间距比值为{}%".format(round(rate_ankle*100,2)))
+    print("脚踝间距与双膝间距比值为{}%".format(round(rate_ankle * 100, 2)))
+    result.append("脚踝间距与双膝间距比值为{}%".format(round(rate_ankle * 100, 2)))
 
-    if rate_ankle > X_ARGUMENT_1 :
+    if rate_ankle > X_ARGUMENT_1:
         print("腿型可能为X型腿")
         result.append("腿型可能为X型腿")
-    elif rate_knee - rate_ankle > O_ARGUMENT_1 :
+    elif rate_knee - rate_ankle > O_ARGUMENT_1:
         print("腿型可能为O型腿")
         result.append("腿型可能为O型腿")
     else:
@@ -303,18 +293,17 @@ def analyse_npy_side(npy_side):
     print("侧面分析内容如下：")
     result.append("侧面分析内容如下：")
 
-
     # for head
-    ear_r_x = data[len(data)-1][0]
-    ear_r_y = data[len(data)-1][1]
+    ear_r_x = data[len(data) - 1][0]
+    ear_r_y = data[len(data) - 1][1]
     shoulder_r_x = data[2][0]
     shoulder_r_y = data[2][1]
     head_forward_angle = angle([shoulder_r_x, shoulder_r_y, ear_r_x, ear_r_y, ], [0, 0, 0, -1])
     head_forward_level = ""
     head_extra_stress = 0
 
-    if head_forward_angle <90 and head_forward_angle > 0:
-        head_extra_stress = round(cal_head_extra_stress(head_forward_angle),3)
+    if head_forward_angle < 90 and head_forward_angle > 0:
+        head_extra_stress = round(cal_head_extra_stress(head_forward_angle), 3)
     if head_forward_angle <= HEAD_FORWARD_ARGUMENT_0:
         head_forward_level = ""
     elif head_forward_angle <= HEAD_FORWARD_ARGUMENT_1 and head_forward_angle > HEAD_FORWARD_ARGUMENT_0:
@@ -324,17 +313,19 @@ def analyse_npy_side(npy_side):
     else:
         head_forward_level = "严重"
     if ear_r_x > shoulder_r_x and head_forward_level != "":
-        print("颈椎分析：头部前倾，角度为{}度，程度{}，颈椎额外承受压力为头部重量{}倍".format(round(head_forward_angle,2),head_forward_level,head_extra_stress))
-        result.append("颈椎分析：头部前倾，角度为{}度，程度{}，颈椎额外承受压力为头部重量{}倍".format(round(head_forward_angle,2),head_forward_level,head_extra_stress))
+        print("颈椎分析：头部前倾，角度为{}度，程度{}，颈椎额外承受压力为头部重量{}倍".format(round(head_forward_angle, 2), head_forward_level,
+                                                              head_extra_stress))
+        result.append("颈椎分析：头部前倾，角度为{}度，程度{}，颈椎额外承受压力为头部重量{}倍".format(round(head_forward_angle, 2), head_forward_level,
+                                                                      head_extra_stress))
     elif ear_r_x < shoulder_r_x and head_forward_level != "":
-        print("颈椎分析：头部后仰，角度为{}度，程度{}，颈椎额外承受压力为头部重量{}倍".format(round(head_forward_angle,2),head_forward_level,head_extra_stress))
-        result.append("颈椎分析：头部后仰，角度为{}度，程度{}，颈椎额外承受压力为头部重量{}倍".format(round(head_forward_angle,2),head_forward_level,head_extra_stress))
+        print("颈椎分析：头部后仰，角度为{}度，程度{}，颈椎额外承受压力为头部重量{}倍".format(round(head_forward_angle, 2), head_forward_level,
+                                                              head_extra_stress))
+        result.append("颈椎分析：头部后仰，角度为{}度，程度{}，颈椎额外承受压力为头部重量{}倍".format(round(head_forward_angle, 2), head_forward_level,
+                                                                      head_extra_stress))
     else:
         print("无头部前倾问题")
         result.append("无头部前倾问题")
 
-
-
     # upper part of body
     up_risk_level = ""
     up_state = ""
@@ -343,7 +334,7 @@ def analyse_npy_side(npy_side):
 
     ankle_r_x = data[10][0]
     ankle_r_y = data[10][1]
-    up_angle =  round(angle([ankle_r_x, ankle_r_y, neck_x, neck_y], [0,0,0,-1]),2)
+    up_angle = round(angle([ankle_r_x, ankle_r_y, neck_x, neck_y], [0, 0, 0, -1]), 2)
     if up_angle >= UP_FORWARD_ARGUMENT_0 and up_angle < UP_FORWARD_ARGUMENT_1:
         up_risk_level = "轻微"
     elif up_angle < UP_FORWARD_ARGUMENT_2 and up_angle >= UP_FORWARD_ARGUMENT_1:
@@ -353,10 +344,10 @@ def analyse_npy_side(npy_side):
     else:
         up_risk_level = ""
 
-    if neck_x < ankle_r_x and up_angle >= HIP_FORWARD_ARGUMENT_0 :
-        up_state = "身体整体后倾，程度{}，身体与竖直方向角度为{}度".format(up_risk_level,up_angle)
-    elif neck_x > ankle_r_x and  up_angle >= HIP_FORWARD_ARGUMENT_0:
-        up_state = "身体整体前倾，程度{}，身体与竖直方向角度为{}度".format(up_risk_level,up_angle)
+    if neck_x < ankle_r_x and up_angle >= HIP_FORWARD_ARGUMENT_0:
+        up_state = "身体整体后倾，程度{}，身体与竖直方向角度为{}度".format(up_risk_level, up_angle)
+    elif neck_x > ankle_r_x and up_angle >= HIP_FORWARD_ARGUMENT_0:
+        up_state = "身体整体前倾，程度{}，身体与竖直方向角度为{}度".format(up_risk_level, up_angle)
     else:
         up_state = "身体整体竖直于地面，无明显站姿前后倾斜"
 
@@ -370,10 +361,10 @@ def analyse_npy_side(npy_side):
     hip_r_y = data[8][1]
     knee_r_x = data[9][0]
     knee_r_y = data[9][1]
-    hip_angle = 180- angle([hip_r_x,hip_r_y,neck_x,neck_y],[hip_r_x,hip_r_y,knee_r_x,knee_r_y])
+    hip_angle = 180 - angle([hip_r_x, hip_r_y, neck_x, neck_y], [hip_r_x, hip_r_y, knee_r_x, knee_r_y])
     if hip_angle >= 90:
-        hip_angle = 180- hip_angle
-    if  hip_angle >= HIP_FORWARD_ARGUMENT_0 and hip_angle < HIP_FORWARD_ARGUMENT_1 :
+        hip_angle = 180 - hip_angle
+    if hip_angle >= HIP_FORWARD_ARGUMENT_0 and hip_angle < HIP_FORWARD_ARGUMENT_1:
         hip_risk_level = "轻微"
     elif hip_angle < HIP_FORWARD_ARGUMENT_2 and hip_angle >= HIP_FORWARD_ARGUMENT_1:
         hip_risk_level = "中度"
@@ -383,10 +374,10 @@ def analyse_npy_side(npy_side):
         hip_risk_level = ""
     if hip_angle > 90:
         hip_angle = 180 - hip_angle
-    if knee_r_x > hip_r_x and hip_angle >= HIP_FORWARD_ARGUMENT_0 :
-        hip_state = "髋部后移，程度{}，躯干与腿部角度为{}度".format(hip_risk_level,round(180-hip_angle,2))
-    elif knee_r_x < hip_r_x and  hip_angle >= HIP_FORWARD_ARGUMENT_0:
-        hip_state = "髋部前移，程度{}，躯干与腿部角度为{}度".format(hip_risk_level,round(180-hip_angle,2))
+    if knee_r_x > hip_r_x and hip_angle >= HIP_FORWARD_ARGUMENT_0:
+        hip_state = "髋部后移，程度{}，躯干与腿部角度为{}度".format(hip_risk_level, round(180 - hip_angle, 2))
+    elif knee_r_x < hip_r_x and hip_angle >= HIP_FORWARD_ARGUMENT_0:
+        hip_state = "髋部前移，程度{}，躯干与腿部角度为{}度".format(hip_risk_level, round(180 - hip_angle, 2))
     else:
         hip_state = "无明显髋部前移或后移"
 
@@ -394,7 +385,7 @@ def analyse_npy_side(npy_side):
     result.append("髋部分析：{}".format(hip_state))
 
     # for knee
-    knee_angle_r = 180-angle([hip_r_x,hip_r_y,knee_r_x,knee_r_y],[knee_r_x,knee_r_y,ankle_r_x,ankle_r_y])
+    knee_angle_r = 180 - angle([hip_r_x, hip_r_y, knee_r_x, knee_r_y], [knee_r_x, knee_r_y, ankle_r_x, ankle_r_y])
     if knee_angle_r > 90:
         knee_angle_r = 180 - knee_angle_r
     knee_forward_risk_level = ""
@@ -411,15 +402,14 @@ def analyse_npy_side(npy_side):
         print("膝盖角度分析：膝盖无明显超伸")
         result.append("膝盖角度分析：膝盖无明显超伸")
     else:
-        print("膝盖角度分析：下肢膝盖超伸角度为{}度，程度{}".format(round(knee_angle_r,2),knee_forward_risk_level))
-        result.append("膝盖角度分析：下肢膝盖超伸角度为{}度，程度{}".format(round(knee_angle_r,2),knee_forward_risk_level))
-
-    return  result
+        print("膝盖角度分析：下肢膝盖超伸角度为{}度，程度{}".format(round(knee_angle_r, 2), knee_forward_risk_level))
+        result.append("膝盖角度分析：下肢膝盖超伸角度为{}度，程度{}".format(round(knee_angle_r, 2), knee_forward_risk_level))
 
+    return result
 
 run_openpose_for_image_front(index)
 run_openpose_for_image_side(index)
 result_front = analyse_npy_front(source_npy_front)
 result_side = analyse_npy_side(source_npy_side)
 result = result_front + result_side
-text_save(save_path,result)
+text_save(save_path, result)

backend/analyse_func.py

@@ -1,10 +1,10 @@
 # -!- coding: utf-8 -!-
-import numpy as np
 import cv2 as cv
 import math
+import numpy as np
 import sys
-from demo_func import run_openpose_for_image_front, run_openpose_for_image_side
 
+from demo_func import run_openpose_for_image_front, run_openpose_for_image_side
 
 def text_save(filename, data):
     file = open(filename, 'w', encoding='utf-8')
@@ -15,7 +15,6 @@ def text_save(filename, data):
     file.close()
     print("保存成功")
 
-
 def angle(v1, v2):
     dx1 = v1[2] - v1[0]
     dy1 = v1[3] - v1[1]
@@ -35,7 +34,6 @@ def angle(v1, v2):
             included_angle = 360 - included_angle
     return included_angle
 
-
 def angle_for_xo(v1, v2):
     dx1 = v1[2] - v1[0]
     dy1 = v1[3] - v1[1]
@@ -47,7 +45,6 @@ def angle_for_xo(v1, v2):
     angle2 = round(angle2 * 180.0 / math.pi, 2)
     return angle1 - angle2
 
-
 def cal_head_extra_stress(angle):
     HEAD_STRESS_ARGUMENT_0 = 11
     HEAD_STRESS_ARGUMENT_1 = 1.02777777777769
@@ -57,7 +54,6 @@ def cal_head_extra_stress(angle):
     return (HEAD_STRESS_ARGUMENT_1 * angle + HEAD_STRESS_ARGUMENT_2 * pow(angle, 2) \
             + HEAD_STRESS_ARGUMENT_3 * pow(angle, 3) + HEAD_STRESS_ARGUMENT_4 * pow(angle, 4)) / HEAD_STRESS_ARGUMENT_0
 
-
 def analyse_npy_front(npy_front, height):
     levels = {
         # (level, threshold)
@@ -88,7 +84,6 @@ def analyse_npy_front(npy_front, height):
     print("正面分析内容如下：")
     result.append("正面分析内容如下：")
 
-
     # h-eye - h-elbow
     h_pixel_eye2elbow = abs(data[14][1] - data[3][1])
     # print("h_pixel{}".format(h_pixel_eye2elbow))
@@ -118,8 +113,8 @@ def analyse_npy_front(npy_front, height):
     elif ear_l_y > ear_r_y:
         head_direction = "右"
     if head_front_level > 0 and head_direction:
-        print("头部分析：头部{}部高，向右倾斜，角度为{}度".format(head_direction, round(head_angle,2)))
-        result.append("头部分析：头部{}部高，向右倾斜，角度为{}度".format(head_direction, round(head_angle,2)))
+        print("头部分析：头部{}部高，向右倾斜，角度为{}度".format(head_direction, round(head_angle, 2)))
+        result.append("头部分析：头部{}部高，向右倾斜，角度为{}度".format(head_direction, round(head_angle, 2)))
     else:
         print("头部分析：头部保持较好平衡")
         result.append("头部分析：头部保持较好平衡")
@@ -173,8 +168,8 @@ def analyse_npy_front(npy_front, height):
         result.append("肩部中心位居中轴，位置良好")
 
     if shoulder_angle_direction:
-        print("肩部向{}倾斜角度为{}度".format(shoulder_angle_direction, round(shoulder_angle,2)))
-        result.append("肩部向{}倾斜角度为{}度".format(shoulder_angle_direction, round(shoulder_angle,2)))
+        print("肩部向{}倾斜角度为{}度".format(shoulder_angle_direction, round(shoulder_angle, 2)))
+        result.append("肩部向{}倾斜角度为{}度".format(shoulder_angle_direction, round(shoulder_angle, 2)))
     else:
         print("肩部无左右倾斜")
         result.append("肩部无左右倾斜")
@@ -222,8 +217,8 @@ def analyse_npy_front(npy_front, height):
         result.append("髋部中心位居中轴，位置良好")
 
     if hip_angle_direction:
-        print("髋部向{}倾斜角度为{}度".format(hip_angle_direction, round(hip_angle,2)))
-        result.append("髋部向{}倾斜角度为{}度".format(hip_angle_direction, round(hip_angle,2)))
+        print("髋部向{}倾斜角度为{}度".format(hip_angle_direction, round(hip_angle, 2)))
+        result.append("髋部向{}倾斜角度为{}度".format(hip_angle_direction, round(hip_angle, 2)))
     else:
         print("髋部角度保持较好平衡")
         result.append("髋部角度保持较好平衡")
@@ -259,7 +254,6 @@ def analyse_npy_front(npy_front, height):
         result.append("腿型基本正常，非X、O型腿")
     return result
 
-
 def analyse_npy_side(npy_side, height):
     levels = {
         "head_forward_angle": [
@@ -307,9 +301,9 @@ def analyse_npy_side(npy_side, height):
         head_forward_direction = "后仰"
     if head_forward_direction:
         print("头部{}角度为{}度，颈椎额外承受压力为头部重量{}倍".format(head_forward_direction, round(head_forward_angle, 2),
-                                                   round(head_extra_stress,2)))
+                                                   round(head_extra_stress, 2)))
         result.append("头部{}角度为{}度，颈椎额外承受压力为头部重量{}倍".format(head_forward_direction, round(head_forward_angle, 2),
-                                                           round(head_extra_stress,2)))
+                                                           round(head_extra_stress, 2)))
     else:
         print("无头部前倾问题")
         result.append("无头部前倾问题")
@@ -330,7 +324,7 @@ def analyse_npy_side(npy_side, height):
         if up_angle <= level[1]:
             break
     up_forward_direction = "后仰" if neck_x < ankle_r_x else "前倾"
-    up_state = "身体{}{}度".format(up_forward_direction, round(up_angle,2))
+    up_state = "身体{}{}度".format(up_forward_direction, round(up_angle, 2))
     print("站姿分析：{}".format(up_state))
     result.append("站姿分析：{}".format(up_state))
     print(up_level)
@@ -353,7 +347,7 @@ def analyse_npy_side(npy_side, height):
     elif hip_r_x < ankle_r_x:
         hip_forward_direction = "后移"
     if hip_forward_direction:
-        hip_state = "髋关节{}{}cm".format(hip_forward_direction, round(hip_offset,2))
+        hip_state = "髋关节{}{}cm".format(hip_forward_direction, round(hip_offset, 2))
     else:
         hip_state = "髋关节正常"
 
@@ -384,7 +378,6 @@ def analyse_npy_side(npy_side, height):
     result.append(knee_forward_risk_level)
     return result
 
-
 def do_analysis(index, height):
     print(index)
     source_npy_front = "pose_temp_data/{}-0.npy".format(index)
@@ -397,11 +390,10 @@ def do_analysis(index, height):
     result = result_front + result_side
     text_save(save_path, result)
 
-
 if __name__ == '__main__':
     heights = [104, 100, 104.5, 124, 107, 120.6, 111, 120, 118, 103.5, 101, 114, 128, 142, 117
         , 181, 105, 183, 150, 177
-    ]
+               ]
     for (i, h) in enumerate(heights):
         try:
             print("{}/{}".format(i + 1, len(heights)))

backend/analyse_jump.py

@@ -1,7 +1,8 @@
-import numpy as np
 import cv2 as cv
 import math
+import numpy as np
 import sys
+
 from demo_func import run_openpose_for_image_front, run_openpose_for_image_side
 
 def angle(v1, v2):
@@ -19,7 +20,7 @@ def angle(v1, v2):
         included_angle = abs(angle1 - angle2)
     else:
         included_angle = abs(angle1) + abs(angle2)
-        #if included_angle > 180:
+        # if included_angle > 180:
         #    included_angle = 360 - included_angle
     return included_angle
 
@@ -30,30 +31,29 @@ def angle1(v1, v2):
     angle2 = abs(round(angle1 * 180.0 / math.pi, 2))
     return angle2
 
-def analyse_npy_side_juanfu(name,num,npy_side):
+def analyse_npy_side_juanfu(name, num, npy_side):
     result = {}
     data = np.load(npy_side)[0]
-    j=0
+    j = 0
     print(len(data))
     # print(data)
-    if(len(data)!=25):
-        
+    if (len(data) != 25):
         return False
-    p="./cap_file/"+name
+    p = "./cap_file/" + name
 
-    file=open("./cap_file/"+name+"/file2.txt","a+")
-    for i in range(0,len(data)):
-        file.write(str(data[i])+"\n")
+    file = open("./cap_file/" + name + "/file2.txt", "a+")
+    for i in range(0, len(data)):
+        file.write(str(data[i]) + "\n")
     file.close()
-    nose_x=data[0][0]
-    nose_y=data[0][1]
-    shoulder_r_x=data[2][0]
-    shoulder_r_y=data[2][1]
-    elbow_r_x=data[3][0]
-    elbow_r_y=data[3][1]
-    wrist_r_x=data[4][0]
-    wrist_r_y=data[4][1]
-    
+    nose_x = data[0][0]
+    nose_y = data[0][1]
+    shoulder_r_x = data[2][0]
+    shoulder_r_y = data[2][1]
+    elbow_r_x = data[3][0]
+    elbow_r_y = data[3][1]
+    wrist_r_x = data[4][0]
+    wrist_r_y = data[4][1]
+
     hip_r_x = data[8][0]
     hip_r_y = data[8][1]
     knee_r_x = data[9][0]
@@ -62,40 +62,39 @@ def analyse_npy_side_juanfu(name,num,npy_side):
     neck_y = data[1][1]
     ankle_r_x = data[10][0]
     ankle_r_y = data[10][1]
-    leg_heigh=data[10][1]
-    neck_hip_angle=round(angle1([neck_x, neck_y, hip_r_x, hip_r_y], [0, 0, 0, -1]), 2)
-    nose_neck_angle=round(angle1([nose_x, nose_y, neck_x, neck_y], [0, 0, 0, -1]), 2)
-    hip_knee_angle=round(angle1([hip_r_x, hip_r_y,knee_r_x, knee_r_y], [0, 0, 0, -1]), 2)
-    knee_ankle_angle = round(angle1([knee_r_x, knee_r_y,ankle_r_x, ankle_r_y], [0, 0, 0, -1]), 2)#洏裔善褐
-    result["neck_hip_angle"]=neck_hip_angle
-    result["nose_neck_angle"]=nose_neck_angle
-    result["hip_knee_angle"]=hip_knee_angle
-    result["knee_ankle_angle"]=knee_ankle_angle
+    leg_heigh = data[10][1]
+    neck_hip_angle = round(angle1([neck_x, neck_y, hip_r_x, hip_r_y], [0, 0, 0, -1]), 2)
+    nose_neck_angle = round(angle1([nose_x, nose_y, neck_x, neck_y], [0, 0, 0, -1]), 2)
+    hip_knee_angle = round(angle1([hip_r_x, hip_r_y, knee_r_x, knee_r_y], [0, 0, 0, -1]), 2)
+    knee_ankle_angle = round(angle1([knee_r_x, knee_r_y, ankle_r_x, ankle_r_y], [0, 0, 0, -1]), 2)  # 洏嚙褒蛛蕭嚙踝蕭
+    result["neck_hip_angle"] = neck_hip_angle
+    result["nose_neck_angle"] = nose_neck_angle
+    result["hip_knee_angle"] = hip_knee_angle
+    result["knee_ankle_angle"] = knee_ankle_angle
 
-    #print(npy_side)
+    # print(npy_side)
     return result
-    
-def analyse_npy_side_pingban(name,num,npy_side):
+
+def analyse_npy_side_pingban(name, num, npy_side):
     result = {}
     data = np.load(npy_side)
-    j=0
-    
-    if(len(data)<15):
-        
+    j = 0
+
+    if (len(data) < 15):
         return False
-    p="./cap_file/"+name
+    p = "./cap_file/" + name
 
-    file=open("./cap_file/"+name+"/file2.txt","a+")
-    for i in range(0,len(data)):
-        file.write(str(data[i])+"\n")
+    file = open("./cap_file/" + name + "/file2.txt", "a+")
+    for i in range(0, len(data)):
+        file.write(str(data[i]) + "\n")
     file.close()
-    shoulder_r_x=data[2][0]
-    shoulder_r_y=data[2][1]
-    elbow_r_x=data[3][0]
-    elbow_r_y=data[3][1]
-    wrist_r_x=data[4][0]
-    wrist_r_y=data[4][1]
-    
+    shoulder_r_x = data[2][0]
+    shoulder_r_y = data[2][1]
+    elbow_r_x = data[3][0]
+    elbow_r_y = data[3][1]
+    wrist_r_x = data[4][0]
+    wrist_r_y = data[4][1]
+
     hip_r_x = data[8][0]
     hip_r_y = data[8][1]
     knee_r_x = data[9][0]
@@ -104,51 +103,50 @@ def analyse_npy_side_pingban(name,num,npy_side):
     neck_y = data[1][1]
     ankle_r_x = data[10][0]
     ankle_r_y = data[10][1]
-    knee_ankle_angle = round(angle1([knee_r_x, knee_r_y,ankle_r_x, ankle_r_y], [0, 0, 0, -1]), 2)#洏裔善褐
-    leg_heigh=data[10][1]
-    neck_hip_angle=round(angle1([neck_x, neck_y, hip_r_x, hip_r_y], [0, 0, 0, -1]), 2)
-    hip_knee_angle=round(angle1([hip_r_x, hip_r_y,knee_r_x, knee_r_y], [0, 0, 0, -1]), 2)
-    s_e_angle=round(angle1([shoulder_r_x,shoulder_r_y,elbow_r_x,elbow_r_y],[]),2)
-    e_w_angle=round(angle1([elbow_r_x,elbow_r_y,wrist_r_x,wrist_r_y],[]),2)
-    hip_heigh=hip_r_y
+    knee_ankle_angle = round(angle1([knee_r_x, knee_r_y, ankle_r_x, ankle_r_y], [0, 0, 0, -1]), 2)  # 洏嚙褒蛛蕭嚙踝蕭
+    leg_heigh = data[10][1]
+    neck_hip_angle = round(angle1([neck_x, neck_y, hip_r_x, hip_r_y], [0, 0, 0, -1]), 2)
+    hip_knee_angle = round(angle1([hip_r_x, hip_r_y, knee_r_x, knee_r_y], [0, 0, 0, -1]), 2)
+    s_e_angle = round(angle1([shoulder_r_x, shoulder_r_y, elbow_r_x, elbow_r_y], []), 2)
+    e_w_angle = round(angle1([elbow_r_x, elbow_r_y, wrist_r_x, wrist_r_y], []), 2)
+    hip_heigh = hip_r_y
 
-    result["knee_ankle_angle"]=knee_ankle_angle
-    result["leg_heigh"]=leg_heigh
-    result["neck_hip_angle"]=neck_hip_angle
-    result["hip_knee_angle"]=hip_knee_angle
-    result["hip_heigh"]=hip_heigh
-    result["s_e_angle"]=s_e_angle
-    result["e_w_angle"]=e_w_angle
+    result["knee_ankle_angle"] = knee_ankle_angle
+    result["leg_heigh"] = leg_heigh
+    result["neck_hip_angle"] = neck_hip_angle
+    result["hip_knee_angle"] = hip_knee_angle
+    result["hip_heigh"] = hip_heigh
+    result["s_e_angle"] = s_e_angle
+    result["e_w_angle"] = e_w_angle
 
-    #print(npy_side)
+    # print(npy_side)
     return result
-    
-def analyse_npy_side_jump(name,num,npy_side):
+
+def analyse_npy_side_jump(name, num, npy_side):
     result = {}
     data = np.load(npy_side)[0]
-    j=0
-    file=open("./cap_file/"+name+"/file2.txt","a+")
-    file.write(str(num)+"\n")
-    for i in range(0,len(data)):
-        file.write(str(data[i])+"\n")
+    j = 0
+    file = open("./cap_file/" + name + "/file2.txt", "a+")
+    file.write(str(num) + "\n")
+    for i in range(0, len(data)):
+        file.write(str(data[i]) + "\n")
     file.close()
-    
-    if(len(data)!=25):
-        
+
+    if (len(data) != 25):
         return False
-    p="./cap_file/"+name
+    p = "./cap_file/" + name
 
-    #file=open("./cap_file/"+name+"/file2.txt","a+")
-    #file.write(str(num)+"\n")
-    #for i in range(0,len(data)):
+    # file=open("./cap_file/"+name+"/file2.txt","a+")
+    # file.write(str(num)+"\n")
+    # for i in range(0,len(data)):
     #    file.write(str(data[i])+"\n")
-    #file.close()
-    #print(data)
+    # file.close()
+    # print(data)
     hip_r_x = data[8][0]
     hip_r_y = data[8][1]
     knee_r_x = data[9][0]
     knee_r_y = data[9][1]
-    #print(head_forward_level)
+    # print(head_forward_level)
     # upper part of body
     up_risk_level = ""
     up_state = ""
@@ -156,46 +154,46 @@ def analyse_npy_side_jump(name,num,npy_side):
     neck_y = data[1][1]
     ankle_r_x = data[10][0]
     ankle_r_y = data[10][1]
-    knee_ankle_angle = round(angle1([knee_r_x, knee_r_y,ankle_r_x, ankle_r_y], [0, 0, 0, -1]), 2)#洏裔善褐
-    leg_heigh=data[10][1]
-    neck_hip_angle=round(angle1([neck_x, neck_y, hip_r_x, hip_r_y], [0, 0, 0, -1]), 2)
-    hip_knee_angle=round(angle1([hip_r_x, hip_r_y,knee_r_x, knee_r_y], [0, 0, 0, -1]), 2)
-    hip_heigh=hip_r_y
+    knee_ankle_angle = round(angle1([knee_r_x, knee_r_y, ankle_r_x, ankle_r_y], [0, 0, 0, -1]), 2)  # 洏嚙褒蛛蕭嚙踝蕭
+    leg_heigh = data[10][1]
+    neck_hip_angle = round(angle1([neck_x, neck_y, hip_r_x, hip_r_y], [0, 0, 0, -1]), 2)
+    hip_knee_angle = round(angle1([hip_r_x, hip_r_y, knee_r_x, knee_r_y], [0, 0, 0, -1]), 2)
+    hip_heigh = hip_r_y
 
-    result["knee_ankle_angle"]=knee_ankle_angle
-    result["leg_heigh"]=leg_heigh
-    result["neck_hip_angle"]=neck_hip_angle
-    result["hip_knee_angle"]=hip_knee_angle
-    result["hip_heigh"]=hip_heigh
+    result["knee_ankle_angle"] = knee_ankle_angle
+    result["leg_heigh"] = leg_heigh
+    result["neck_hip_angle"] = neck_hip_angle
+    result["hip_knee_angle"] = hip_knee_angle
+    result["hip_heigh"] = hip_heigh
 
     return result
-    
-def analyse_npy_side_gaotaitui(name,num,npy_side):
+
+def analyse_npy_side_gaotaitui(name, num, npy_side):
     result = {}
     print(npy_side)
     data = np.load(npy_side)[0]
-    j=0
+    j = 0
     print("analyse_npy_side_gaotaitui------")
     print(data)
-    file=open("./cap_file/"+name+"/file2.txt","a+")
-    file.write(str(num)+"\n")
-    for i in range(0,len(data)):
-        file.write(str(data[i])+"\n")
+    file = open("./cap_file/" + name + "/file2.txt", "a+")
+    file.write(str(num) + "\n")
+    for i in range(0, len(data)):
+        file.write(str(data[i]) + "\n")
     file.close()
-    if(len(data)!=25):
+    if (len(data) != 25):
         return False
-    p="./cap_file/"+name
-    #print(data)
+    p = "./cap_file/" + name
+    # print(data)
     hip_r_x = data[8][0]
     hip_r_y = data[8][1]
     knee_r_x = data[9][0]
     knee_r_y = data[9][1]
-    
+
     hip_l_x = data[11][0]
     hip_l_y = data[11][1]
     knee_l_x = data[12][0]
     knee_l_y = data[12][1]
-    #print(head_forward_level)
+    # print(head_forward_level)
     # upper part of body
     up_risk_level = ""
     up_state = ""
@@ -205,52 +203,49 @@ def analyse_npy_side_gaotaitui(name,num,npy_side):
     ankle_r_y = data[10][1]
     ankle_l_x = data[13][0]
     ankle_l_y = data[13][1]
-    knee_ankle_r_angle = round(angle1([knee_r_x, knee_r_y,ankle_r_x, ankle_r_y], [0, 0, 0, -1]), 2)#洏裔善褐
-    knee_ankle_l_angle = round(angle1([knee_l_x, knee_l_y,ankle_l_x, ankle_l_y], [0, 0, 0, -1]), 2)#洏裔善褐
-    
-    hip_knee_r_angle=round(angle1([hip_r_x, hip_r_y,knee_r_x, knee_r_y], [0, 0, 0, -1]), 2)
-    hip_knee_l_angle=round(angle1([hip_l_x, hip_l_y,knee_l_x, knee_l_y], [0, 0, 0, -1]), 2)
-    neck_hip_angle=round(angle1([neck_x, neck_y, hip_r_x, hip_r_y], [0, 0, 0, -1]), 2)
-    #hip_knee_angle=round(angle1([hip_r_x, hip_r_y,knee_r_x, knee_r_y], [0, 0, 0, -1]), 2)
-    hip_heigh=hip_r_y
+    knee_ankle_r_angle = round(angle1([knee_r_x, knee_r_y, ankle_r_x, ankle_r_y], [0, 0, 0, -1]), 2)  # 洏嚙褒蛛蕭嚙踝蕭
+    knee_ankle_l_angle = round(angle1([knee_l_x, knee_l_y, ankle_l_x, ankle_l_y], [0, 0, 0, -1]), 2)  # 洏嚙褒蛛蕭嚙踝蕭
 
-    result["knee_ankle_r_angle"]=knee_ankle_r_angle
-    result["knee_ankle_l_angle"]=knee_ankle_l_angle
-    result["hip_knee_r_angle"]=hip_knee_r_angle
-    result["hip_knee_l_angle"]=hip_knee_l_angle
-    result["r_d"]=abs(0-float(hip_knee_r_angle)+float(knee_ankle_r_angle))
-    result["l_d"]=abs(0+float(knee_ankle_l_angle)-float(hip_knee_l_angle))
-   
+    hip_knee_r_angle = round(angle1([hip_r_x, hip_r_y, knee_r_x, knee_r_y], [0, 0, 0, -1]), 2)
+    hip_knee_l_angle = round(angle1([hip_l_x, hip_l_y, knee_l_x, knee_l_y], [0, 0, 0, -1]), 2)
+    neck_hip_angle = round(angle1([neck_x, neck_y, hip_r_x, hip_r_y], [0, 0, 0, -1]), 2)
+    # hip_knee_angle=round(angle1([hip_r_x, hip_r_y,knee_r_x, knee_r_y], [0, 0, 0, -1]), 2)
+    hip_heigh = hip_r_y
+
+    result["knee_ankle_r_angle"] = knee_ankle_r_angle
+    result["knee_ankle_l_angle"] = knee_ankle_l_angle
+    result["hip_knee_r_angle"] = hip_knee_r_angle
+    result["hip_knee_l_angle"] = hip_knee_l_angle
+    result["r_d"] = abs(0 - float(hip_knee_r_angle) + float(knee_ankle_r_angle))
+    result["l_d"] = abs(0 + float(knee_ankle_l_angle) - float(hip_knee_l_angle))
 
-    
     return result
-    
-def analyse_npy_side_shendun(name,num,npy_side):
+
+def analyse_npy_side_shendun(name, num, npy_side):
     result = {}
     data = np.load(npy_side)[0]
-    j=0
-    
+    j = 0
 
-    file=open("./cap_file/"+name+"/file2.txt","a+")
-    file.write(str(num)+"\n")
-    
-    if(len(data)!=25):
+    file = open("./cap_file/" + name + "/file2.txt", "a+")
+    file.write(str(num) + "\n")
+
+    if (len(data) != 25):
         return False
-    for i in range(0,len(data)):
-        file.write(str(data[i])+"\n")
+    for i in range(0, len(data)):
+        file.write(str(data[i]) + "\n")
     file.close()
-    p="./cap_file/"+name
-    #print(data)
+    p = "./cap_file/" + name
+    # print(data)
     hip_r_x = data[8][0]
     hip_r_y = data[8][1]
     knee_r_x = data[9][0]
     knee_r_y = data[9][1]
-    
+
     hip_l_x = data[11][0]
     hip_l_y = data[11][1]
     knee_l_x = data[12][0]
     knee_l_y = data[12][1]
-    #print(head_forward_level)
+    # print(head_forward_level)
     # upper part of body
     up_risk_level = ""
     up_state = ""
@@ -260,22 +255,19 @@ def analyse_npy_side_shendun(name,num,npy_side):
     ankle_r_y = data[10][1]
     ankle_l_x = data[13][0]
     ankle_l_y = data[13][1]
-    knee_ankle_r_angle = round(angle1([knee_r_x, knee_r_y,ankle_r_x, ankle_r_y], [0, 0, 0, -1]), 2)#洏裔善褐
-    knee_ankle_l_angle = round(angle1([knee_l_x, knee_l_y,ankle_l_x, ankle_l_y], [0, 0, 0, -1]), 2)#洏裔善褐
-    
-    hip_knee_r_angle=round(angle1([hip_r_x, hip_r_y,knee_r_x, knee_r_y], [0, 0, 0, -1]), 2)
-    hip_knee_l_angle=round(angle1([hip_l_x, hip_l_y,knee_l_x, knee_l_y], [0, 0, 0, -1]), 2)
-    neck_hip_angle=round(angle1([neck_x, neck_y, hip_r_x, hip_r_y], [0, 0, 0, -1]), 2)
-    #hip_knee_angle=round(angle1([hip_r_x, hip_r_y,knee_r_x, knee_r_y], [0, 0, 0, -1]), 2)
-    hip_heigh=hip_r_y
+    knee_ankle_r_angle = round(angle1([knee_r_x, knee_r_y, ankle_r_x, ankle_r_y], [0, 0, 0, -1]), 2)  # 洏嚙褒蛛蕭嚙踝蕭
+    knee_ankle_l_angle = round(angle1([knee_l_x, knee_l_y, ankle_l_x, ankle_l_y], [0, 0, 0, -1]), 2)  # 洏嚙褒蛛蕭嚙踝蕭
+
+    hip_knee_r_angle = round(angle1([hip_r_x, hip_r_y, knee_r_x, knee_r_y], [0, 0, 0, -1]), 2)
+    hip_knee_l_angle = round(angle1([hip_l_x, hip_l_y, knee_l_x, knee_l_y], [0, 0, 0, -1]), 2)
+    neck_hip_angle = round(angle1([neck_x, neck_y, hip_r_x, hip_r_y], [0, 0, 0, -1]), 2)
+    # hip_knee_angle=round(angle1([hip_r_x, hip_r_y,knee_r_x, knee_r_y], [0, 0, 0, -1]), 2)
+    hip_heigh = hip_r_y
 
-    result["knee_ankle_r_angle"]=knee_ankle_r_angle
-    result["knee_ankle_l_angle"]=knee_ankle_l_angle
-    result["hip_knee_r_angle"]=hip_knee_r_angle
-    result["hip_knee_l_angle"]=hip_knee_l_angle
-    result["r_d"]=abs(float(result["hip_knee_r_angle"])-float(result["knee_ankle_r_angle"]))
-    
-   
+    result["knee_ankle_r_angle"] = knee_ankle_r_angle
+    result["knee_ankle_l_angle"] = knee_ankle_l_angle
+    result["hip_knee_r_angle"] = hip_knee_r_angle
+    result["hip_knee_l_angle"] = hip_knee_l_angle
+    result["r_d"] = abs(float(result["hip_knee_r_angle"]) - float(result["knee_ankle_r_angle"]))
 
-    
     return result

backend/bak/demo_func_bak.py

@@ -1,20 +1,17 @@
+import copy
 import cv2
 import matplotlib.pyplot as plt
-import copy
 import numpy as np
-from PIL import Image
 import os
+from PIL import Image
 
 from src import util
 from src.body import Body
+from video_pose.data.prepare_data_2d_custom import custom
 # from src.infer_video_d2 import infer_image,load_predictor
 from video_pose.inference.infer_video_d2 import run_3d
-from video_pose.data.prepare_data_2d_custom import custom
 from video_pose.run import run_video_reconstruction
 
-
-
-
 def run_3d_for_video(original_video_path):
     run_3d(original_video_path)
     video_analyzed_path = original_video_path + ".npz"
@@ -26,7 +23,6 @@ def run_3d_for_video(original_video_path):
     print(reconstructed_video_path)
     return reconstructed_video_path
 
-
 def run_openpose_for_image_front(index):
     PNG_OR_NOT = True
     # hand_estimation = Hand('model/hand_pose_model.pth')
@@ -68,7 +64,6 @@ def run_openpose_for_image_front(index):
     # plt.savefig('pose_processed_images/{}-1-result.png'.format(index))
     # plt.show()
 
-
 def run_openpose_for_image_side(index):
     PNG_OR_NOT = True
     body_estimation = Body('model/body_pose_model.pth')
@@ -108,41 +103,39 @@ def run_openpose_for_image_side(index):
     # plt.savefig('pose_processed_images/{}-1-result.png'.format(index))
     # plt.show()
 
-
-def run_openpose_for_normal(index,name):
+def run_openpose_for_normal(index, name):
     print(index)
-    
+
     # 输入index和原来一样，就是序号，你可以更改一下输入或者输出，方便你们处理
 
     # 判断是否是PNG格式，如果是，增添对alpha通道的处理
     PNG_OR_NOT = True
     # 调用的模型路径，无需更改
     body_estimation = Body('model/body_pose_model.pth')
-    #原图像的位置，需要更改 ！！
-    p='./capture_image/'+name
-    test_image = p+'/capture_image{}.png'.format(index)
+    # 原图像的位置，需要更改 ！！
+    p = './capture_image/' + name
+    test_image = p + '/capture_image{}.png'.format(index)
     if os.path.exists(test_image) is False:
         PNG_OR_NOT = False
         # 原图像的位置，需要更改 ！！
-        test_image = p+'/capture_image{}.jpg'.format(index)
+        test_image = p + '/capture_image{}.jpg'.format(index)
     oriImg = cv2.imread(test_image)  # B,G,R order
     im = cv2.imread(test_image, cv2.IMREAD_UNCHANGED)
 
     candidate, subset = body_estimation(oriImg)
     # 坐标点数值保存路径，需要更改！！
-    np.save(p+'/capture_image{}-1.png'.format(index), candidate)
-    print(p+'/capture_image{}.png'.format(index))
+    np.save(p + '/capture_image{}-1.png'.format(index), candidate)
+    print(p + '/capture_image{}.png'.format(index))
 
     # 处理
     canvas = copy.deepcopy(oriImg)
     canvas = util.draw_bodypose(canvas, candidate, subset)
-    #if PNG_OR_NOT == True:
-        #alpha_value_new = np.reshape(im[:,:,3],(im.shape[0],im.shape[1],1))
-        #canvas = np.c_[canvas, alpha_value_new]
+    # if PNG_OR_NOT == True:
+    # alpha_value_new = np.reshape(im[:,:,3],(im.shape[0],im.shape[1],1))
+    # canvas = np.c_[canvas, alpha_value_new]
 
     # 结果图片的保存路径，需要更改 ！！
-    cv2.imwrite(p+'/capture_image_result{}.png'.format(index), canvas, [int(cv2.IMWRITE_PNG_COMPRESSION), 9])
-
+    cv2.imwrite(p + '/capture_image_result{}.png'.format(index), canvas, [int(cv2.IMWRITE_PNG_COMPRESSION), 9])
 
 if __name__ == '__main__':
     # run_openpose_for_normal(1)

backend/bak/flask_api_bak.py

@@ -1,14 +1,14 @@
+import base64
 import json
 import os
-
+from PIL import Image
 from flask import Flask, request
 from flask_cors import *
-from PIL import Image
-import base64
 
-from analyse_func import  do_analysis
+from analyse_func import do_analysis
 from classify import api_dyna
-from db_project import get_status,process_db,lookup_data ,add_new_data_dyna,add_new_data,create_table_dynamic,create_table,create_db
+from db_project import get_status, process_db, lookup_data, add_new_data_dyna, add_new_data, create_table_dynamic, \
+    create_table, create_db
 
 # str_res=""
 # a dict contains the txt
@@ -17,14 +17,10 @@ li_last = []
 # store dic_data's json format
 bJson = ""
 global_index = [6]
-global_video_index=0
+global_video_index = 0
 li_return = []
 img_return = []
-flag_run=False
-
-
-
-
+flag_run = False
 
 ##jump
 ##result["jump_power"]["level"]/["path"]/["val"]
@@ -32,7 +28,7 @@ flag_run=False
 ##result["landing_position"]["level"]/["path"]["val"]
 ##juanfu
 ## result["head_power"]["level"]/["path"]/["val"]
-#result["core_strength"]["level"]/["path"]
+# result["core_strength"]["level"]/["path"]
 ##result["body_stability"]["level"]/["val"]
 
 ##pingban
@@ -50,8 +46,6 @@ result["leg_strength"]["level"]/["path"]
 result["standard"]["level"]/["path"]
 """
 
-
-
 # read txt file and change it to the list format
 def read_pose_processes_txt_result():
     file_dir = "pose_processed_txt"  # the file's path
@@ -81,25 +75,25 @@ def read_pose_processes_txt_result():
                 i = i + 1  # i是头部分析那行
                 dic_1["name"] = "头部分析"
                 dic_1["value"] = lines[i][5:len(lines[i]) - 1]
-                i = i + 1 # 头部数字级别
+                i = i + 1  # 头部数字级别
                 dic_1["level"] = int(lines[i].strip())
                 li.append(dic_1)
                 dic_2 = {}
-                i += 2 #第六行 i=6
+                i += 2  # 第六行 i=6
                 dic_2["name"] = "肩部分析"
-                #num_risk_1 = int(lines[i + 1].strip())
+                # num_risk_1 = int(lines[i + 1].strip())
                 dic_2["value"] = lines[i][:-1] + " " + lines[(i + 1)][:-1]
                 num_risk_2 = int(lines[i + 2].strip())
                 dic_2["level"] = int(num_risk_2)
                 li.append(dic_2)
-                i += 4 #i=10
+                i += 4  # i=10
                 dic_3 = {}
                 dic_3["name"] = "髋部分析"
                 dic_3["value"] = lines[i][:-1] + " " + lines[(i + 1)][:-1]
                 num_risk_1 = int(lines[i + 2].strip())
-                #num_risk_2 = int(lines[i + 3].strip())
+                # num_risk_2 = int(lines[i + 3].strip())
                 dic_3["level"] = int(max(num_risk_1, 0))
-                i += 4 #i=14
+                i += 4  # i=14
                 dic_4 = {"name": "腿型分析", "value": lines[i][:-1]}
                 li.append(dic_3)
                 li.append(dic_4)
@@ -145,16 +139,16 @@ def read_pose_processes_txt_result():
     global li_return
     li_return = dict_total[(str(global_index[0]) + ".txt")]
 
-
 def run_the_process(index: int = 0, height: float = 110):
     # sys.path.append("../analyse_func.py")
     # import analyse_func
-    #print(global_index[0])
+    # print(global_index[0])
     do_analysis(global_index[0], height)
+
 # 把list转化为可以直接返回的json格式
 
 def list_to_json(li):
-    #print(li)
+    # print(li)
 
     aJson = json.dumps(li, ensure_ascii=False)
     print(aJson)
@@ -165,115 +159,115 @@ def list_to_json(li):
     # print(dict1)
     return aJson
     # print(dic_data)
+
 app = Flask(__name__)
+
 def save_file(filed):
     base = "pose_source_images/"
     filename = filed.filename
     index = global_index[0]
-    #global_index[0] = index
+    # global_index[0] = index
     suffix = ".png"
     if filename.find("jpg") > 0:
         suffix = ".jpg"
     elif filename.find("png") > 0:
         suffix = ".png"
     save_name = ""
-    print(str(index)+"index")
-    print(filename+"filename")
+    print(str(index) + "index")
+    print(filename + "filename")
     if "front" in filename:
         save_name = base + str(index) + str("-0") + suffix
-        print(filename+"filename_front")
-        print(save_name+"savename_front")
+        print(filename + "filename_front")
+        print(save_name + "savename_front")
     elif "right" in filename:
         save_name = base + str(index) + str("-1") + suffix
-        print(filename+"filename_right")
-        print(save_name+"savename_right")
-    else :
+        print(filename + "filename_right")
+        print(save_name + "savename_right")
+    else:
         print("nope")
     filed.save(save_name)
     print(filename)
     print(save_name)
-#运行动态分析的入口
-def run_video(mode,num,ana=1):
-    q="./video/"+str(num)+".mp4"
-    re= api_dyna(q,mode,ana)
+
+# 运行动态分析的入口
+def run_video(mode, num, ana=1):
+    q = "./video/" + str(num) + ".mp4"
+    re = api_dyna(q, mode, ana)
     process_db(num)
     return re
 
-def lookup_video(num,ana=0):
-    q="./video/"+str(num)+".mp4"
-    mode=lookup_data(num)
-    re= api_dyna(q,mode,ana)
+def lookup_video(num, ana=0):
+    q = "./video/" + str(num) + ".mp4"
+    mode = lookup_data(num)
+    re = api_dyna(q, mode, ana)
     process_db(num)
     return re
 
 ###send a video and return a id marked the video (identical)
-@app.route('/get_videoid',methods=['POST'])
+@app.route('/get_videoid', methods=['POST'])
 @cross_origin()
 def get_videoid():
-    dict_video_=request.form.to_dict()
-    dict_video_mp4=request.files.to_dict()
+    dict_video_ = request.form.to_dict()
+    dict_video_mp4 = request.files.to_dict()
 
-    ff=dict_video_mp4["video"]
-    mode=dict_video_["mode"]
+    ff = dict_video_mp4["video"]
+    mode = dict_video_["mode"]
 
     global global_video_index
-    global_video_index=int(add_new_data_dyna(mode))
+    global_video_index = int(add_new_data_dyna(mode))
 
-    base="./video/"
-    filename=base+str(global_video_index)+".mp4"
+    base = "./video/"
+    filename = base + str(global_video_index) + ".mp4"
 
     ff.save(filename)
 
-    return {"id":str(global_video_index)}
-
+    return {"id": str(global_video_index)}
 
-@app.route('/get_video_data',methods=['POST'])
+@app.route('/get_video_data', methods=['POST'])
 @cross_origin()
 def get_video_data():
-    dict_video_=request.form.to_dict()
+    dict_video_ = request.form.to_dict()
     global global_video_index
-    global_video_index=int(dict_video_["id"])
-    mode=str(lookup_data(global_video_index))#get database mode
-    re=run_video(mode,global_video_index,1)
+    global_video_index = int(dict_video_["id"])
+    mode = str(lookup_data(global_video_index))  # get database mode
+    re = run_video(mode, global_video_index, 1)
     return re
 
-@app.route('/get_video_status',methods=['POST'])
+@app.route('/get_video_status', methods=['POST'])
 @cross_origin()
 def get_video_status():
-    dict_video_=request.form.to_dict()
+    dict_video_ = request.form.to_dict()
     global global_video_index
-    global_video_index=int(dict_video_["id"])
-    bool=get_status(global_video_index)==1
-    return {"state":get_status(global_video_index)==1}
-
+    global_video_index = int(dict_video_["id"])
+    bool = get_status(global_video_index) == 1
+    return {"state": get_status(global_video_index) == 1}
 
-
-@app.route('/query_video',methods=['POST'])
+@app.route('/query_video', methods=['POST'])
 @cross_origin()
 def query_video():
-    dict_video_=request.form.to_dict()
+    dict_video_ = request.form.to_dict()
     print(dict_video_)
-    global  global_video_index
-    global_video_index=int(dict_video_["id"])
+    global global_video_index
+    global_video_index = int(dict_video_["id"])
     print(global_video_index)
-    return lookup_video(global_video_index,0)
+    return lookup_video(global_video_index, 0)
 
-@app.route('/get_3D',methods=['POST'])
+@app.route('/get_3D', methods=['POST'])
 @cross_origin()
 def get_3D():
     return "404"
 
-@app.route('/query_3D',methods=['POST'])
+@app.route('/query_3D', methods=['POST'])
 @cross_origin()
 def query_3D():
     return 404
 
-#result={xx:xx,xx:xx}
-#result={power:{val:xx,path:xx},strength:{val:xx,path:xx}}
+# result={xx:xx,xx:xx}
+# result={power:{val:xx,path:xx},strength:{val:xx,path:xx}}
 
-#1.锻炼建议
-#2.发送视频
-#3.
+# 1.锻炼建议
+# 2.发送视频
+# 3.
 
 @app.route('/send_form', methods=['POST'])
 @cross_origin()
@@ -282,31 +276,29 @@ def send_form():
 
     fileList = request.files.to_dict()
     global global_index
-    global_index[0]=int(add_new_data())
+    global_index[0] = int(add_new_data())
     for filed in fileList.values():
         save_file(filed)
     height = float(text_data["height"]) if text_data["height"] != "undefined" else None
     run_the_process(global_index[0], height)
     process_db(int(global_index[0]))
     return {
-        "id":global_index[0],
+        "id": global_index[0],
         "status": True if height is not None else False
     }
 
-
 def get_base64(path):
     with open(path, "rb") as f:
         base64_data = base64.b64encode(f.read())
         return base64_data.decode()
 
-
 @app.route('/get_result', methods=['POST'])
 @cross_origin()
 def run_project():
     id = request.form.to_dict()["id"]
     print(id)
     base = "pose_processed_images"
-    global_index[0]=int(id)
+    global_index[0] = int(id)
     read_pose_processes_txt_result()
     print(base + "/" + str(global_index[0]) + "-0-result.png")
     return {
@@ -315,15 +307,13 @@ def run_project():
         "data": list_to_json(li_return),
     }
 
-
-
 if __name__ == '__main__':
     # read_pose_processes_txt_result()
     # print(list_to_json(li_return))
 
-    #app.run()
-    app.run(host="0.0.0.0", port=8090,debug=True)
-    #app.run(host="0.0.0.0", port=8090)
+    # app.run()
+    app.run(host="0.0.0.0", port=8090, debug=True)
+    # app.run(host="0.0.0.0", port=8090)
     CORS(app, resouces=r'/*')
-    #run_video("gaotaitui",27,0)
-    lookup_video(41,0)
+    # run_video("gaotaitui",27,0)
+    lookup_video(41, 0)

backend/bak/video_to_photo_bak.py

@@ -1,39 +1,37 @@
-import os
-import math
-
-import  cv2
-from demo_func import run_openpose_for_normal
-from PIL import Image
-from analyse_jump import  analyse_npy_side_jump,analyse_npy_side_pingban,analyse_npy_side_juanfu,analyse_npy_side_gaotaitui,analyse_npy_side_shendun
+import cv2
 import glob
-import time
+import math
 import os
+import os
+import time
+from PIL import Image
 
-
-
+from analyse_jump import analyse_npy_side_jump, analyse_npy_side_pingban, analyse_npy_side_juanfu, \
+    analyse_npy_side_gaotaitui, analyse_npy_side_shendun
+from demo_func import run_openpose_for_normal
 
 def split_video(path):
     video_path = path
     os.path.split(path)
-    tur=os.path.split(path)
-    print(tur[len(tur)-1].split('.')[0])
-    name=tur[len(tur)-1].split('.')[0]
-    if os.path.exists("./capture_image/"+name)==False:
-        os.mkdir("./capture_image/"+name)
+    tur = os.path.split(path)
+    print(tur[len(tur) - 1].split('.')[0])
+    name = tur[len(tur) - 1].split('.')[0]
+    if os.path.exists("./capture_image/" + name) == False:
+        os.mkdir("./capture_image/" + name)
     cap = cv2.VideoCapture(video_path)
     FPS = cap.get(5)
     print(FPS)
     c = 1
     frameRate = 1
-    if os.path.exists("./capture_image/"+name)==False:
-        os.mkdir("./capture_image/"+name)
+    if os.path.exists("./capture_image/" + name) == False:
+        os.mkdir("./capture_image/" + name)
 
     while (True):
 
         ret, frame = cap.read()
         if ret:
             if (c % frameRate == 0):
-                cv2.imwrite("./capture_image/"+name+"/capture_image" + str(c) + '.png', frame)
+                cv2.imwrite("./capture_image/" + name + "/capture_image" + str(c) + '.png', frame)
 
             c += 1
             cv2.waitKey(0)
@@ -42,39 +40,39 @@ def split_video(path):
     cap.release()
     return c
 
-def process_photo(name,c=334):
+def process_photo(name, c=334):
     print(name)
-    for i in range(1,c):
-        run_openpose_for_normal(i,name)
+    for i in range(1, c):
+        run_openpose_for_normal(i, name)
 
 def gather_video(path):
     video_path = path
 
     os.path.split(path)
-    tur=os.path.split(path)
-    print(tur[len(tur)-1].split('.')[0])
-    name=tur[len(tur)-1].split('.')[0]
+    tur = os.path.split(path)
+    print(tur[len(tur) - 1].split('.')[0])
+    name = tur[len(tur) - 1].split('.')[0]
 
     cap = cv2.VideoCapture(video_path)
     fps = cap.get(5)
-    num_frames = len(os.listdir(r'./capture_image/'+name))
+    num_frames = len(os.listdir(r'./capture_image/' + name))
     print(num_frames)
     img_array = []
-    im=Image.open("./capture_image/"+name+"/capture_image_result1.png")
+    im = Image.open("./capture_image/" + name + "/capture_image_result1.png")
 
-    #img_width = 720
-    #img_height = 1280
-    k=0
+    # img_width = 720
+    # img_height = 1280
+    k = 0
     for i in range(num_frames + 1):
-        filename = "./capture_image/"+name+"/capture_image_result" + str(i) + ".png"
-        k+=1
+        filename = "./capture_image/" + name + "/capture_image_result" + str(i) + ".png"
+        k += 1
         img = cv2.imread(filename)
 
         if img is None:
             continue
         img_array.append(img)
     print(k)
-    save_name="./video/video_a/"+name+".mp4"
+    save_name = "./video/video_a/" + name + ".mp4"
     out = cv2.VideoWriter(save_name, cv2.VideoWriter_fourcc(*"mp4v"), fps, im.size)
 
     for i in range(len(img_array)):
@@ -82,415 +80,403 @@ def gather_video(path):
     out.release()
     im.close()
 
-
-def do_analysis(a_path,video_path,mode):
+def do_analysis(a_path, video_path, mode):
     path = video_path
     os.path.split(path)
-    tur=os.path.split(path)
-    name=tur[len(tur)-1].split('.')[0]
-
-    if os.path.exists("./cap_file/"+name)==False:
-        os.mkdir("./cap_file/"+name)
-        #ticks=time.time()
-    source_npy_side=a_path
-    output=open("./cap_file/"+name+"/file2"+".txt","w")
+    tur = os.path.split(path)
+    name = tur[len(tur) - 1].split('.')[0]
+
+    if os.path.exists("./cap_file/" + name) == False:
+        os.mkdir("./cap_file/" + name)
+        # ticks=time.time()
+    source_npy_side = a_path
+    output = open("./cap_file/" + name + "/file2" + ".txt", "w")
     output.close()
-    result={}
-    p='./capture_image/'+name+"/"
-    lenF=int(len(glob.glob(p+'*.png'))/2)
+    result = {}
+    p = './capture_image/' + name + "/"
+    lenF = int(len(glob.glob(p + '*.png')) / 2)
     print(lenF)
-    for i in range(1,lenF):#run_openpose_for_normal(i)
-        if(mode=="jump"):
-            result_side = analyse_npy_side_jump(name,i,source_npy_side.format(i))
-        elif (mode=="pingban"):
-            result_side = analyse_npy_side_pingban(name,i,source_npy_side.format(i))
-        elif(mode=="juanfu"):
-            result_side = analyse_npy_side_juanfu(name,i,source_npy_side.format(i))
-        elif(mode=="gaotaitui"):
-            result_side =analyse_npy_side_gaotaitui(name,i,source_npy_side.format(i))
-        elif(mode=="shendun"):
-            result_side =analyse_npy_side_shendun(name,i,source_npy_side.format(i))
-
-        if(result_side!=False):
-            result[i]=result_side
-
-
-    output=open("./cap_file/"+name+"/file1.txt","w")
+    for i in range(1, lenF):  # run_openpose_for_normal(i)
+        if (mode == "jump"):
+            result_side = analyse_npy_side_jump(name, i, source_npy_side.format(i))
+        elif (mode == "pingban"):
+            result_side = analyse_npy_side_pingban(name, i, source_npy_side.format(i))
+        elif (mode == "juanfu"):
+            result_side = analyse_npy_side_juanfu(name, i, source_npy_side.format(i))
+        elif (mode == "gaotaitui"):
+            result_side = analyse_npy_side_gaotaitui(name, i, source_npy_side.format(i))
+        elif (mode == "shendun"):
+            result_side = analyse_npy_side_shendun(name, i, source_npy_side.format(i))
+
+        if (result_side != False):
+            result[i] = result_side
+
+    output = open("./cap_file/" + name + "/file1.txt", "w")
     for re in result:
-        print(re,file=output)
-        for(name,value) in result[re].items():
-            print(name,value,sep=',',file=output)
-        print("kv-over",file=output)
+        print(re, file=output)
+        for (name, value) in result[re].items():
+            print(name, value, sep=',', file=output)
+        print("kv-over", file=output)
     output.close()
 
-
 def read2(fileName="./cap_file/file1.txt"):
-    file=open(fileName)
-    db={}
-    key=file.readline().strip()
+    file = open(fileName)
+    db = {}
+    key = file.readline().strip()
 
-    while(key  != "kv-over\n"):
+    while (key != "kv-over\n"):
         if not key:
             break
-        person={}
-        field=file.readline().strip()
+        person = {}
+        field = file.readline().strip()
         while field != "kv-over":
-            name, value= field.split(",")
-            person[name]=value.strip()
-            field=file.readline().strip()
-            if(field=="kv-over\n"):
+            name, value = field.split(",")
+            person[name] = value.strip()
+            field = file.readline().strip()
+            if (field == "kv-over\n"):
                 break
-        db[key]=person
+        db[key] = person
         key = file.readline().strip()
-    print("db==== " , db)
+    print("db==== ", db)
     return db
 
-
-
 def ignore_data_pingban(result):
-    result_ig=result
-    k="0"
-    last={}
-    max_index=1
+    result_ig = result
+    k = "0"
+    last = {}
+    max_index = 1
     for re in result:
-        if k=="0":
-            k=re
+        if k == "0":
+            k = re
         else:
-            if(max_index<int(re)):
-                max_index=int(re)
-
-    max_angle=abs(float(result_ig[k]["s_e_angle"])-float(result_ig[k]["e_w_angle"]))
-    max_num=int(k)
+            if (max_index < int(re)):
+                max_index = int(re)
 
-    less_angle=100
-    less_num=1
+    max_angle = abs(float(result_ig[k]["s_e_angle"]) - float(result_ig[k]["e_w_angle"]))
+    max_num = int(k)
 
-    max_less_a=1
-    max_less_num=k
+    less_angle = 100
+    less_num = 1
 
+    max_less_a = 1
+    max_less_num = k
 
     for i in result_ig:
-        if (int(i)>=max_index/3 and int(i)<=max_index/3*2):
-            val=math.fabs( math.fabs(float(result_ig[i]["hip_knee_angle"]))-math.fabs(float(result_ig[i]["knee_ankle_angle"])))
-            an_=abs(float(result_ig[i]["s_e_angle"])-float(result_ig[i]["e_w_angle"]))
-            if(math.fabs( math.fabs(float(result_ig[i]["hip_knee_angle"]))-math.fabs(float(result_ig[i]["knee_ankle_angle"])))<less_angle):
-                less_angle=val
-                less_num=int(i)
-            if(an_<max_angle):
-                max_angle=an_
-                max_num=i
-            if(math.fabs( math.fabs(float(result_ig[i]["hip_knee_angle"]))-math.fabs(float(result_ig[i]["knee_ankle_angle"])))>max_less_a):
-                max_less_an=val
-                max_less_num=int(i)
-    re={}
-    re["1"]={}
-    re["1"]["num"]=max_num
-    re["1"]["val"]=max_angle
-    re["2"]={}
-    re["2"]["num"]=less_num
-    re["2"]["val"]=less_angle
-    re["3"]={}
-    re["3"]["num"]=max_less_num
-    re["3"]["val"]=max_less_a
-    #1表示手肘和肩膀最小角度
-    #2表示最小的身体角度差
-    #3表示最大的身体角度差
-    #1用来说明手部力量 3-2说明稳定性 2说明最好的状态 3说明最差的状态
+        if (int(i) >= max_index / 3 and int(i) <= max_index / 3 * 2):
+            val = math.fabs(
+                math.fabs(float(result_ig[i]["hip_knee_angle"])) - math.fabs(float(result_ig[i]["knee_ankle_angle"])))
+            an_ = abs(float(result_ig[i]["s_e_angle"]) - float(result_ig[i]["e_w_angle"]))
+            if (math.fabs(math.fabs(float(result_ig[i]["hip_knee_angle"])) - math.fabs(
+                    float(result_ig[i]["knee_ankle_angle"]))) < less_angle):
+                less_angle = val
+                less_num = int(i)
+            if (an_ < max_angle):
+                max_angle = an_
+                max_num = i
+            if (math.fabs(math.fabs(float(result_ig[i]["hip_knee_angle"])) - math.fabs(
+                    float(result_ig[i]["knee_ankle_angle"]))) > max_less_a):
+                max_less_an = val
+                max_less_num = int(i)
+    re = {}
+    re["1"] = {}
+    re["1"]["num"] = max_num
+    re["1"]["val"] = max_angle
+    re["2"] = {}
+    re["2"]["num"] = less_num
+    re["2"]["val"] = less_angle
+    re["3"] = {}
+    re["3"]["num"] = max_less_num
+    re["3"]["val"] = max_less_a
+    # 1表示手肘和肩膀最小角度
+    # 2表示最小的身体角度差
+    # 3表示最大的身体角度差
+    # 1用来说明手部力量 3-2说明稳定性 2说明最好的状态 3说明最差的状态
     print(re)
-    return result_ig,re
+    return result_ig, re
 
 def ignore_data_juanfu(result):
-    result_ig=result
-    k="0"
-    last={}
-    max_index=1
+    result_ig = result
+    k = "0"
+    last = {}
+    max_index = 1
     for re in result:
-        if k=="0":
-            k=re
+        if k == "0":
+            k = re
         else:
-            if(max_index<int(re)):
-                max_index=int(re)
+            if (max_index < int(re)):
+                max_index = int(re)
 
-    max_angle=result[k]["nose_neck_angle"]
-    max_num=int(k)
+    max_angle = result[k]["nose_neck_angle"]
+    max_num = int(k)
 
-    less_h_k_a=result[k]["hip_knee_angle"]
-    less_h_k_num=k
+    less_h_k_a = result[k]["hip_knee_angle"]
+    less_h_k_num = k
 
-    max_h_k_a=result[k]["hip_knee_angle"]
-    max_h_k_num=k
+    max_h_k_a = result[k]["hip_knee_angle"]
+    max_h_k_num = k
 
-    less_k_a_a=result[k]["knee_ankle_angle"]
-    less_k_a_num=k
+    less_k_a_a = result[k]["knee_ankle_angle"]
+    less_k_a_num = k
 
-    max_k_a_a=result[k]["knee_ankle_angle"]
-    max_k_a_num=k
+    max_k_a_a = result[k]["knee_ankle_angle"]
+    max_k_a_num = k
 
-    max_neck_hip_a=result[k]["neck_hip_angle"]
-    max_neck_hip_num=k
+    max_neck_hip_a = result[k]["neck_hip_angle"]
+    max_neck_hip_num = k
 
     for i in result_ig:
-        if(float(result_ig[i]["nose_neck_angle"])>float(max_angle)):
-            max_num=int(i)
-            max_angle=result_ig[i]["nose_neck_angle"]
-
-        if(float(result_ig[i]["hip_knee_angle"])>float(max_h_k_a)):
-            max_h_k_num=int(i)
-            max_h_k_a=result_ig[i]["hip_knee_angle"]
-
-        if(float(result_ig[i]["knee_ankle_angle"])>float(max_k_a_a)):
-            max_k_a_num=int(i)
-            max_k_a_a=result_ig[i]["knee_ankle_angle"]
-
-        if(float(result_ig[i]["neck_hip_angle"])>float(max_neck_hip_a)):
-            max_neck_hip_num=int(i)
-            max_neck_hip_a=result_ig[i]["neck_hip_angle"]
-
-
-        if(float(result_ig[i]["hip_knee_angle"])<float(less_h_k_a)):
-            less_h_k_num=int(i)
-            less_h_k_a=result_ig[i]["hip_knee_angle"]
-
-        if(float(result_ig[i]["knee_ankle_angle"])<float(less_k_a_a)):
-            less_k_a_num=int(i)
-            less_k_a_a=result_ig[i]["knee_ankle_angle"]
-
-
-    re={}
-    re["1"]={}
-    re["1"]["num"]=max_num
-    re["1"]["val"]=max_angle
-    re["2"]={}
-    re["2"]["num"]=max_neck_hip_num
-    re["2"]["val"]=max_neck_hip_a
-    re["3"]={}
-    re["3"]["num"]=float(max_h_k_a)-float(less_h_k_a)
-    re["3"]["num1"]=max_h_k_num
-    re["3"]["num2"]=less_h_k_num
-    re["3"]["val1"]=max_h_k_a
-    re["3"]["val2"]=less_h_k_a
-    re["4"]={}
-    re["4"]["num"]=float(max_k_a_a)-float(less_k_a_a)
-    re["4"]["num1"]=max_k_a_num
-    re["4"]["num2"]=less_k_a_num
-    re["4"]["val1"]=max_k_a_a
-    re["4"]["val2"]=less_k_a_a
-    #1表示头部和脖子角度
-    #2表示脖子和臀部角度
-    #3表示hip-knee的极差 稳定性
-    #4表示knee-ankle的极差 稳定性
-    #3的数据误差太大 待用性待考察
+        if (float(result_ig[i]["nose_neck_angle"]) > float(max_angle)):
+            max_num = int(i)
+            max_angle = result_ig[i]["nose_neck_angle"]
+
+        if (float(result_ig[i]["hip_knee_angle"]) > float(max_h_k_a)):
+            max_h_k_num = int(i)
+            max_h_k_a = result_ig[i]["hip_knee_angle"]
+
+        if (float(result_ig[i]["knee_ankle_angle"]) > float(max_k_a_a)):
+            max_k_a_num = int(i)
+            max_k_a_a = result_ig[i]["knee_ankle_angle"]
+
+        if (float(result_ig[i]["neck_hip_angle"]) > float(max_neck_hip_a)):
+            max_neck_hip_num = int(i)
+            max_neck_hip_a = result_ig[i]["neck_hip_angle"]
+
+        if (float(result_ig[i]["hip_knee_angle"]) < float(less_h_k_a)):
+            less_h_k_num = int(i)
+            less_h_k_a = result_ig[i]["hip_knee_angle"]
+
+        if (float(result_ig[i]["knee_ankle_angle"]) < float(less_k_a_a)):
+            less_k_a_num = int(i)
+            less_k_a_a = result_ig[i]["knee_ankle_angle"]
+
+    re = {}
+    re["1"] = {}
+    re["1"]["num"] = max_num
+    re["1"]["val"] = max_angle
+    re["2"] = {}
+    re["2"]["num"] = max_neck_hip_num
+    re["2"]["val"] = max_neck_hip_a
+    re["3"] = {}
+    re["3"]["num"] = float(max_h_k_a) - float(less_h_k_a)
+    re["3"]["num1"] = max_h_k_num
+    re["3"]["num2"] = less_h_k_num
+    re["3"]["val1"] = max_h_k_a
+    re["3"]["val2"] = less_h_k_a
+    re["4"] = {}
+    re["4"]["num"] = float(max_k_a_a) - float(less_k_a_a)
+    re["4"]["num1"] = max_k_a_num
+    re["4"]["num2"] = less_k_a_num
+    re["4"]["val1"] = max_k_a_a
+    re["4"]["val2"] = less_k_a_a
+    # 1表示头部和脖子角度
+    # 2表示脖子和臀部角度
+    # 3表示hip-knee的极差 稳定性
+    # 4表示knee-ankle的极差 稳定性
+    # 3的数据误差太大 待用性待考察
     print(re)
-    return result_ig,re
-
+    return result_ig, re
 
 def ignore_data_gaotaitui(result):
-
-    result_ig=result
-    k="0"
-    last={}
-    max_index=1
+    result_ig = result
+    k = "0"
+    last = {}
+    max_index = 1
     for re in result:
-        if k=="0":
-            k=re
+        if k == "0":
+            k = re
         else:
-            if(max_index<int(re)):
-                max_index=int(re)
+            if (max_index < int(re)):
+                max_index = int(re)
 
-
-    max_leg_knee_r_angle=result_ig[k]["knee_ankle_r_angle"]
-    max_leg_knee_l_angle=result_ig[k]["knee_ankle_l_angle"]
-    max_leg_knee_angle_num=int(k)
-    panduan={}
+    max_leg_knee_r_angle = result_ig[k]["knee_ankle_r_angle"]
+    max_leg_knee_l_angle = result_ig[k]["knee_ankle_l_angle"]
+    max_leg_knee_angle_num = int(k)
+    panduan = {}
     for i in result_ig:
-        if(float(result_ig[i]["l_d"])<30):
-            if(float(result_ig[i]["knee_ankle_r_angle"])>120 and float(result_ig[i]["hip_knee_r_angle"])<90):
-                panduan[i]=result_ig[i]["r_d"]
-    f=0
-    #ma=0
-    ma=panduan[k]
-    key_k=k
+        if (float(result_ig[i]["l_d"]) < 30):
+            if (float(result_ig[i]["knee_ankle_r_angle"]) > 120 and float(result_ig[i]["hip_knee_r_angle"]) < 90):
+                panduan[i] = result_ig[i]["r_d"]
+    f = 0
+    # ma=0
+    ma = panduan[k]
+    key_k = k
     for key in panduan:
-        if f==0:
-            ma=panduan[key]
-            f=1
-            key_k=key
-        elif float(ma)>float(panduan[key]):
-            key_k=key
-            ma=panduan[key]
-
-    re={}
-    re["1"]={}
-    re["1"]["num"]=key_k
-    re["1"]["val"]=ma
-    re["1"]["hip_knee_r_angle"]=result_ig[key_k]["hip_knee_r_angle"]
-    re["1"]["knee_ankle_r_angle"]=result_ig[key_k]["knee_ankle_r_angle"]
-
-
-    return result_ig,re
+        if f == 0:
+            ma = panduan[key]
+            f = 1
+            key_k = key
+        elif float(ma) > float(panduan[key]):
+            key_k = key
+            ma = panduan[key]
+
+    re = {}
+    re["1"] = {}
+    re["1"]["num"] = key_k
+    re["1"]["val"] = ma
+    re["1"]["hip_knee_r_angle"] = result_ig[key_k]["hip_knee_r_angle"]
+    re["1"]["knee_ankle_r_angle"] = result_ig[key_k]["knee_ankle_r_angle"]
+
+    return result_ig, re
 
 def ignore_data_jump(result):
-    result_ig=result
-    k="0"
-    last={}
-    max_index=1
+    result_ig = result
+    k = "0"
+    last = {}
+    max_index = 1
     for re in result:
-        if k=="0":
-            k=re
+        if k == "0":
+            k = re
         else:
-            if(max_index<int(re)):
-                max_index=int(re)
+            if (max_index < int(re)):
+                max_index = int(re)
 
     print(result_ig)
-    max_leg_knee_angle=result_ig[k]["knee_ankle_angle"]
-    max_leg_knee_angle_num=int(k)
-    less_angle=100
-    less_num=1
-    ankle=result_ig[k]["leg_heigh"]
-    ankle_num=int(k)
-    max_leg_knee_hip_knee=math.fabs(float(result_ig[k]["hip_knee_angle"])-float(result_ig[k]["knee_ankle_angle"]))
-    max_l_h_num=1
-    hip_hei_init=float(result_ig[k]["hip_heigh"])
-    #print("max")
-    #print(max_index)
-    #print(hip_hei_init)
+    max_leg_knee_angle = result_ig[k]["knee_ankle_angle"]
+    max_leg_knee_angle_num = int(k)
+    less_angle = 100
+    less_num = 1
+    ankle = result_ig[k]["leg_heigh"]
+    ankle_num = int(k)
+    max_leg_knee_hip_knee = math.fabs(float(result_ig[k]["hip_knee_angle"]) - float(result_ig[k]["knee_ankle_angle"]))
+    max_l_h_num = 1
+    hip_hei_init = float(result_ig[k]["hip_heigh"])
+    # print("max")
+    # print(max_index)
+    # print(hip_hei_init)
 
     for i in result_ig:
-        if int(i)<=max_index/5:
-            if float(max_leg_knee_angle)<float(result_ig[i]["knee_ankle_angle"]):
-                max_leg_knee_angle_num=int(i)
-                max_leg_knee_angle=result_ig[i]["knee_ankle_angle"]
-        if int(i)<=max_index/2 and int(i)>max_index/3:
-            val=math.fabs( math.fabs(float(result_ig[i]["hip_knee_angle"]))-math.fabs(float(result_ig[i]["knee_ankle_angle"])))
-            if(math.fabs( math.fabs(float(result_ig[i]["hip_knee_angle"]))-math.fabs(float(result_ig[i]["knee_ankle_angle"])))<less_angle):
-                less_angle=val
-                less_num=int(i)
-
-        if int(i)<=max_index and int(i)>(max_index/3):
-
-            if(float(result_ig[i]["hip_heigh"]))<(float(hip_hei_init)) and max_l_h_num!=1:
+        if int(i) <= max_index / 5:
+            if float(max_leg_knee_angle) < float(result_ig[i]["knee_ankle_angle"]):
+                max_leg_knee_angle_num = int(i)
+                max_leg_knee_angle = result_ig[i]["knee_ankle_angle"]
+        if int(i) <= max_index / 2 and int(i) > max_index / 3:
+            val = math.fabs(
+                math.fabs(float(result_ig[i]["hip_knee_angle"])) - math.fabs(float(result_ig[i]["knee_ankle_angle"])))
+            if (math.fabs(math.fabs(float(result_ig[i]["hip_knee_angle"])) - math.fabs(
+                    float(result_ig[i]["knee_ankle_angle"]))) < less_angle):
+                less_angle = val
+                less_num = int(i)
+
+        if int(i) <= max_index and int(i) > (max_index / 3):
+
+            if (float(result_ig[i]["hip_heigh"])) < (float(hip_hei_init)) and max_l_h_num != 1:
                 break
-            if (math.fabs(math.fabs(float(result_ig[i]["hip_knee_angle"]))-math.fabs(float(result_ig[i]["knee_ankle_angle"]))))>max_leg_knee_hip_knee and math.fabs(float(result_ig[i]["leg_heigh"])-float(ankle))<=20:
-                max_leg_knee_hip_knee=math.fabs(float(result_ig[i]["hip_knee_angle"])-math.fabs(float(result_ig[i]["knee_ankle_angle"])))
-                max_l_h_num=int(i)
+            if (math.fabs(math.fabs(float(result_ig[i]["hip_knee_angle"])) - math.fabs(
+                    float(result_ig[i]["knee_ankle_angle"])))) > max_leg_knee_hip_knee and math.fabs(
+                    float(result_ig[i]["leg_heigh"]) - float(ankle)) <= 20:
+                max_leg_knee_hip_knee = math.fabs(
+                    float(result_ig[i]["hip_knee_angle"]) - math.fabs(float(result_ig[i]["knee_ankle_angle"])))
+                max_l_h_num = int(i)
 
             #    ankle=result_ig[i]["leg_heigh"]
             #    ankle_num=int(i)
-            #elif ankle!=1 and int(i)>=(le/3*2):
+            # elif ankle!=1 and int(i)>=(le/3*2):
             #    break
 
     print(max_leg_knee_angle)
     print(max_leg_knee_angle_num)
     print(less_num)
     print(less_angle)
-    #print(ankle)
-    #print(ankle_num)
+    # print(ankle)
+    # print(ankle_num)
     print(max_leg_knee_hip_knee)
     print(max_l_h_num)
-    re={}
-    re["1"]={}
-    re["1"]["num"]=max_leg_knee_angle_num
-    re["1"]["val"]=max_leg_knee_angle
-    re["2"]={}
-    re["2"]["num"]=less_num
-    re["2"]["val"]=less_angle
-    re["3"]={}
-    re["3"]["num"]=max_l_h_num
-    re["3"]["val"]=max_leg_knee_hip_knee
+    re = {}
+    re["1"] = {}
+    re["1"]["num"] = max_leg_knee_angle_num
+    re["1"]["val"] = max_leg_knee_angle
+    re["2"] = {}
+    re["2"]["num"] = less_num
+    re["2"]["val"] = less_angle
+    re["3"] = {}
+    re["3"]["num"] = max_l_h_num
+    re["3"]["val"] = max_leg_knee_hip_knee
     print(re)
 
-
-    return result_ig,re
-
+    return result_ig, re
 
 def ignore_data_shendun(result):
-    result_ig=result
-    k="0"
-    last={}
-    max_index=1
+    result_ig = result
+    k = "0"
+    last = {}
+    max_index = 1
     for re in result:
-        if k=="0":
-            k=re
+        if k == "0":
+            k = re
         else:
-            if(max_index<int(re)):
-                max_index=int(re)
+            if (max_index < int(re)):
+                max_index = int(re)
 
-
-    max_d_num=k
-    max_d=result[k]["r_d"]
+    max_d_num = k
+    max_d = result[k]["r_d"]
     for i in result_ig:
-        if(float(result_ig[i]["r_d"])>float(max_d)):
-            max_d_num=int(i)
-            max_d= result_ig[i]["r_d"]
-
+        if (float(result_ig[i]["r_d"]) > float(max_d)):
+            max_d_num = int(i)
+            max_d = result_ig[i]["r_d"]
 
-    re={}
-    re["1"]={}
-    re["1"]["num"]=max_d_num
-    re["1"]["val"]=max_d
+    re = {}
+    re["1"] = {}
+    re["1"]["num"] = max_d_num
+    re["1"]["val"] = max_d
 
-    #1最大的深蹲角度
-    return result_ig,re
+    # 1最大的深蹲角度
+    return result_ig, re
 
-def write1(result,name):
-    output=open(name,"w")
+def write1(result, name):
+    output = open(name, "w")
     for re in result:
-        print(re,file=output)
-        for(name,value) in result[re].items():
-            print(name,value,sep=',',file=output)
-        print("kv-over",file=output)
+        print(re, file=output)
+        for (name, value) in result[re].items():
+            print(name, value, sep=',', file=output)
+        print("kv-over", file=output)
     output.close()
     return name
 
-
-def run(video_path,mode,need_split):
-    path=video_path
+def run(video_path, mode, need_split):
+    path = video_path
     os.path.split(path)
-    tur=os.path.split(path)
-    if(mode=="pingban"):
+    tur = os.path.split(path)
+    if (mode == "pingban"):
         print(need_split)
-    name=tur[len(tur)-1].split('.')[0]
-    if(need_split==1):
-        c=split_video(video_path)
-        process_photo(name,c)
+    name = tur[len(tur) - 1].split('.')[0]
+    if (need_split == 1):
+        c = split_video(video_path)
+        process_photo(name, c)
         gather_video(video_path)
-    do_analysis("./capture_image/"+name+"/capture_image{}-1.png.npy",video_path,mode)
+    do_analysis("./capture_image/" + name + "/capture_image{}-1.png.npy", video_path, mode)
 
-    switch = {'jump': ignore_data_jump,                # 注意此处不要加括号
+    switch = {'jump': ignore_data_jump,  # 注意此处不要加括号
               'pingban': ignore_data_pingban,
               'juanfu': ignore_data_juanfu,
-              'shendun':ignore_data_shendun,
-              'gaotaitui':ignore_data_gaotaitui,
+              'shendun': ignore_data_shendun,
+              'gaotaitui': ignore_data_gaotaitui,
               }
-    choice = mode                        # 获取选择
-    result,re=switch.get(choice,  ignore_data_jump)(read2("./cap_file/"+name+"/file1.txt"))
+    choice = mode  # 获取选择
+    result, re = switch.get(choice, ignore_data_jump)(read2("./cap_file/" + name + "/file1.txt"))
     # 执行对应的函数，如果没有就执行默认的函数
-    #result,re=ignore_data_juanfu(read2("./cap_file/"+name+"/file1.txt"))
-    write1(result,"./cap_file/"+name+"/file3.txt")
-    write1(re,"./cap_file/"+name+"-file.txt")
-    #write1()
+    # result,re=ignore_data_juanfu(read2("./cap_file/"+name+"/file1.txt"))
+    write1(result, "./cap_file/" + name + "/file3.txt")
+    write1(re, "./cap_file/" + name + "-file.txt")
+    # write1()
     return re
 
 if __name__ == '__main__':
     video_path = "./video/shendun.mp4"
-    #path=video_path
-    #os.path.split(path)
-    #tur=os.path.split(path)
-    #name=tur[len(tur)-1].split('.')[0]
-
-
-    #c=split_video(video_path)
-    #process_photo(name,c)
-    #gather_video(video_path)
-    #do_analysis("./capture_image/"+name+"/capture_image{}-1.png.npy",video_path,"shendun")
-    #result,re=ignore_data_shendun(read2("./cap_file/"+name+"/file1.txt"))
-    #write1(result,"./cap_file/"+name+"/file3.txt")
-    #write1(re,"./cap_file/"+name+"-file.txt")
-    #write1()
-    #print(re)
-    re=run(video_path,"shendun",0)
-    #参数一 video_path
-    #参数二 shendun jump gaotaitui juanfu pingban
-    #第三个参数为1 表示需要视频切割
-
-
+    # path=video_path
+    # os.path.split(path)
+    # tur=os.path.split(path)
+    # name=tur[len(tur)-1].split('.')[0]
+
+    # c=split_video(video_path)
+    # process_photo(name,c)
+    # gather_video(video_path)
+    # do_analysis("./capture_image/"+name+"/capture_image{}-1.png.npy",video_path,"shendun")
+    # result,re=ignore_data_shendun(read2("./cap_file/"+name+"/file1.txt"))
+    # write1(result,"./cap_file/"+name+"/file3.txt")
+    # write1(re,"./cap_file/"+name+"-file.txt")
+    # write1()
+    # print(re)
+    re = run(video_path, "shendun", 0)
+    # 参数一 video_path
+    # 参数二 shendun jump gaotaitui juanfu pingban
+    # 第三个参数为1 表示需要视频切割

backend/classify.py

@@ -1,16 +1,14 @@
 import base64
-
-from video_to_photo import run
 import os
-import advice
 
+import advice
+from video_to_photo import run
 
 def get_base64(path):
     with open(path, "rb") as f:
         base64_data = base64.b64encode(f.read())
         return base64_data.decode()
 
-
 def classify_jump(name, re):
     q = "./capture_image/" + name
     qqq = os.path.abspath(q)
@@ -84,7 +82,6 @@ def classify_jump(name, re):
     # res.append(d)
     return {"analyzed_data": res, "advice": result["advice"]}
 
-
 def classify_juanfu(name, re):
     q = "./capture_image/" + name
     qqq = os.path.abspath(q)
@@ -156,7 +153,6 @@ def classify_juanfu(name, re):
     # res.append(d)
     return {"analyzed_data": res, "advice": result["advice"]}
 
-
 def classify_pingban(name, re):
     q = "./capture_image/" + name
     qqq = os.path.abspath(q)
@@ -229,7 +225,6 @@ def classify_pingban(name, re):
     # res.append(d)
     return {"analyzed_data": res, "advice": result["advice"]}
 
-
 def classify_shendun(name, re):
     q = "./capture_image/" + name
     qqq = os.path.abspath(q)
@@ -262,7 +257,6 @@ def classify_shendun(name, re):
     # res.append(d)
     return {"analyzed_data": res, "advice": result["advice"]}
 
-
 def classify_gaotaitui(name, re):
     q = "./capture_image/" + name
     qqq = os.path.abspath(q)
@@ -296,7 +290,6 @@ def classify_gaotaitui(name, re):
     # res.append(d)
     return {"analyzed_data": res, "advice": result["advice"]}
 
-
 def classify_result(video_path, re, mode):
     # video_path = "./video/demo1.mp4"
     path = video_path
@@ -315,11 +308,9 @@ def classify_result(video_path, re, mode):
     print(power)
     return power
 
-
 def api_dyna(path, mode, ana):
     return classify_result(path, run(path, mode, ana), mode)
 
-
 if __name__ == '__main__':
     # path="./video/demo1.mp4"
     # api_dyna(path,"jump",0)

backend/config.py

@@ -1,6 +1,7 @@
 # config.py
-import os
 import gevent.monkey
+import os
+
 gevent.monkey.patch_all()
 
 import multiprocessing
@@ -16,4 +17,4 @@ daemon = True
 # 启动的进程数
 workers = multiprocessing.cpu_count()
 worker_class = 'gevent'
-x_forwarded_for_header = 'X-FORWARDED-FOR'
+x_forwarded_for_header = 'X-FORWARDED-FOR'

backend/db_project.py

@@ -1,6 +1,6 @@
 import pymysql
 
-ip_ad="106.15.1.178"
+ip_ad = "106.15.1.178"
 
 db = pymysql.connect(host=ip_ad,
                      user="root",
@@ -14,19 +14,20 @@ def create_db():
     db = pymysql.connect(host=ip_ad,
                          user="root",
                          password="root",
-                         #port=3306,  # 端口
-                         #charset='utf8')
+                         # port=3306,  # 端口
+                         # charset='utf8')
                          )
 
     cursor = db.cursor()
     # 使用 execute() 方法执行 SQL，如果表存在则删除
-    #cursor.execute("DROP TABLE IF EXISTS pose")
+    # cursor.execute("DROP TABLE IF EXISTS pose")
 
     # 使用预处理语句创建表
     cursor.execute("create database if not exists pose")
     db.commit()
     db.close()
     cursor.close()
+
 def create_table():
     db = pymysql.connect(host=ip_ad,
                          user="root",
@@ -37,7 +38,6 @@ def create_table():
 
     cursor = db.cursor()
 
-
     sql = """CREATE TABLE pose(
              id  INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
              status INT)
@@ -51,6 +51,7 @@ def create_table():
     print(version)
     db.commit()
     db.close()
+
 def create_table_dynamic():
     db = pymysql.connect(host=ip_ad,
                          user="root",
@@ -78,6 +79,7 @@ def create_table_dynamic():
     print(version)
     db.commit()
     db.close()
+
 def add_new_data():
     db = pymysql.connect(host=ip_ad,
                          user="root",
@@ -88,12 +90,12 @@ def add_new_data():
 
     cursor = db.cursor()
 
-    insert_table_sql="""INSERT INTO `pose` (`status`) VALUES ( '0')"""
+    insert_table_sql = """INSERT INTO `pose` (`status`) VALUES ( '0')"""
     cursor.execute(insert_table_sql)
     db.commit()
     db.close()
     cursor.close()
-    return_id=cursor.lastrowid
+    return_id = cursor.lastrowid
     return return_id
 
 def add_new_data_dyna(modes):
@@ -106,12 +108,12 @@ def add_new_data_dyna(modes):
 
     cursor = db.cursor()
 
-    insert_table_sql="""INSERT INTO `video` (`status`,`mode`) VALUES ( '%d','%s')""" %(0,modes)
+    insert_table_sql = """INSERT INTO `video` (`status`,`mode`) VALUES ( '%d','%s')""" % (0, modes)
     cursor.execute(insert_table_sql)
     db.commit()
     db.close()
     cursor.close()
-    return_id=cursor.lastrowid
+    return_id = cursor.lastrowid
     return return_id
 
 def lookup_data(id):
@@ -122,13 +124,14 @@ def lookup_data(id):
                          database="pose",
                          charset='utf8')
     cursor = db.cursor()
-    sql="select mode from video where id=%d"  %id
+    sql = "select mode from video where id=%d" % id
     cursor.execute(sql)
-    result=cursor.fetchone()
+    result = cursor.fetchone()
     db.commit()
     cursor.close()
     db.close()
     return result[0]
+
 def process_db(id):
     db = pymysql.connect(host=ip_ad,
                          user="root",
@@ -136,7 +139,7 @@ def process_db(id):
                          port=3306,  # 端口
                          database="pose",
                          charset='utf8')
-    sql = "update video set status = 1 where id = %d " %id
+    sql = "update video set status = 1 where id = %d " % id
 
     cursor = db.cursor()
     cursor.execute(sql)
@@ -152,14 +155,14 @@ def get_status(id):
                          port=3306,  # 端口
                          database="pose",
                          charset='utf8')
-    sql="select status from video where id =%d"%id
-    cursor=db.cursor()
+    sql = "select status from video where id =%d" % id
+    cursor = db.cursor()
     cursor.execute(sql)
-    result=cursor.fetchone()
+    result = cursor.fetchone()
     cursor.close()
     db.close()
 
     return result
+
 if __name__ == '__main__':
     print(add_new_data_dyna("jump"))
-

backend/demo.py

@@ -1,19 +1,19 @@
+import copy
 import cv2
 import matplotlib.pyplot as plt
-import copy
 import numpy as np
 
 from src import util
 from src.body import Body
 
 body_estimation = Body('model/body_pose_model.pth')
-#hand_estimation = Hand('model/hand_pose_model.pth')
+# hand_estimation = Hand('model/hand_pose_model.pth')
 
 test_image = 'pose_source_images/4-0.png'
 oriImg = cv2.imread(test_image)  # B,G,R order
 candidate, subset = body_estimation(oriImg)
 print(candidate)
-np.save('pose_temp_data/4-0.npy',candidate)
+np.save('pose_temp_data/4-0.npy', candidate)
 canvas = copy.deepcopy(oriImg)
 canvas = util.draw_bodypose(canvas, candidate, subset)
 # detect hand

backend/demo_camera.py

@@ -1,5 +1,5 @@
-import cv2
 import copy
+import cv2
 import numpy as np
 import torch
 
@@ -27,17 +27,16 @@ while True:
 
     all_hand_peaks = []
     for x, y, w, is_left in hands_list:
-        peaks = hand_estimation(oriImg[y:y+w, x:x+w, :])
-        peaks[:, 0] = np.where(peaks[:, 0]==0, peaks[:, 0], peaks[:, 0]+x)
-        peaks[:, 1] = np.where(peaks[:, 1]==0, peaks[:, 1], peaks[:, 1]+y)
+        peaks = hand_estimation(oriImg[y:y + w, x:x + w, :])
+        peaks[:, 0] = np.where(peaks[:, 0] == 0, peaks[:, 0], peaks[:, 0] + x)
+        peaks[:, 1] = np.where(peaks[:, 1] == 0, peaks[:, 1], peaks[:, 1] + y)
         all_hand_peaks.append(peaks)
 
     canvas = util.draw_handpose(canvas, all_hand_peaks)
 
-    cv2.imshow('demo', canvas)#一个窗口用以显示原视频
+    cv2.imshow('demo', canvas)  # 一个窗口用以显示原视频
     if cv2.waitKey(1) & 0xFF == ord('q'):
         break
 
 cap.release()
 cv2.destroyAllWindows()
-

backend/demo_func.py

@@ -1,20 +1,19 @@
+import copy
 import cv2
 import matplotlib.pyplot as plt
-import copy
 import numpy as np
-from PIL import Image
 import os
+from PIL import Image
 
+import src1
 from src import util
 from src.body import Body
 from src1.torch_openpose import torch_openpose
-import src1
+from video_pose.data.prepare_data_2d_custom import custom
 # from src.infer_video_d2 import infer_image,load_predictor
 from video_pose.inference.infer_video_d2 import run_3d
-from video_pose.data.prepare_data_2d_custom import custom
 from video_pose.run import run_video_reconstruction
 
-
 def run_3d_for_video(original_video_path):
     run_3d(original_video_path)
     video_analyzed_path = original_video_path + ".npz"
@@ -26,7 +25,6 @@ def run_3d_for_video(original_video_path):
     print(reconstructed_video_path)
     # return reconstructed_video_path
 
-
 def run_openpose_for_image_front(index):
     PNG_OR_NOT = True
     # hand_estimation = Hand('model/hand_pose_model.pth')
@@ -48,7 +46,6 @@ def run_openpose_for_image_front(index):
         canvas = np.c_[canvas, alpha_value_new]
     cv2.imwrite('pose_processed_images/{}-0-result.png'.format(index), canvas, [int(cv2.IMWRITE_PNG_COMPRESSION), 9])
 
-
 # def run_openpose_for_image_front(index):
 #     PNG_OR_NOT = True
 #     # hand_estimation = Hand('model/hand_pose_model.pth')
@@ -116,7 +113,6 @@ def run_openpose_for_image_side(index):
         canvas = np.c_[canvas, alpha_value_new]
     cv2.imwrite('pose_processed_images/{}-1-result.png'.format(index), canvas, [int(cv2.IMWRITE_PNG_COMPRESSION), 9])
 
-
 # def run_openpose_for_image_side(index):
 #     PNG_OR_NOT = True
 #     body_estimation = Body('model/body_pose_model.pth')
@@ -156,7 +152,7 @@ def run_openpose_for_image_side(index):
 #     # plt.savefig('pose_processed_images/{}-1-result.png'.format(index))
 #     # plt.show()
 
-def run_openpose_for_normal(index,name):
+def run_openpose_for_normal(index, name):
     PNG_OR_NOT = True
     tp = torch_openpose('body_25')
     p = './capture_image/' + name
@@ -233,7 +229,5 @@ if __name__ == '__main__':
     # input_video = "video/15.mp4"
     # run_3d_for_video(input_video)
 
-    run_openpose_for_normal(1,"20")
+    run_openpose_for_normal(1, "20")
     # run_openpose_for_normal1(2, "20")
-
-

backend/demo_video.py

@@ -1,22 +1,19 @@
-import copy
-import numpy as np
-import cv2
-import os
-import json
-
 # video file processing setup
 # from: https://stackoverflow.com/a/61927951
 import argparse
+import copy
+import cv2
+import json
+import numpy as np
+import os
 import subprocess
 from typing import NamedTuple
 
-
 class FFProbeResult(NamedTuple):
     return_code: int
     json: str
     error: str
 
-
 def ffprobe(file_path) -> FFProbeResult:
     command_array = ["ffprobe",
                      "-v", "quiet",
@@ -29,7 +26,6 @@ def ffprobe(file_path) -> FFProbeResult:
                          json=result.stdout,
                          error=result.stderr)
 
-
 # openpose setup
 from src import util
 from src.body import Body
@@ -47,9 +43,9 @@ def process_frame(frame, body=True, hands=True):
         hands_list = util.handDetect(candidate, subset, frame)
         all_hand_peaks = []
         for x, y, w, is_left in hands_list:
-            peaks = hand_estimation(frame[y:y+w, x:x+w, :])
-            peaks[:, 0] = np.where(peaks[:, 0]==0, peaks[:, 0], peaks[:, 0]+x)
-            peaks[:, 1] = np.where(peaks[:, 1]==0, peaks[:, 1], peaks[:, 1]+y)
+            peaks = hand_estimation(frame[y:y + w, x:x + w, :])
+            peaks[:, 0] = np.where(peaks[:, 0] == 0, peaks[:, 0], peaks[:, 0] + x)
+            peaks[:, 1] = np.where(peaks[:, 1] == 0, peaks[:, 1], peaks[:, 1] + y)
             all_hand_peaks.append(peaks)
         canvas = util.draw_handpose(canvas, all_hand_peaks)
     return canvas
@@ -60,7 +56,7 @@ import ffmpeg
 
 # open specified video
 parser = argparse.ArgumentParser(
-        description="Process a video annotating poses detected.")
+    description="Process a video annotating poses detected.")
 parser.add_argument('file', type=str, help='Video file location to process.')
 parser.add_argument('--no_hands', action='store_true', help='No hand pose')
 parser.add_argument('--no_body', action='store_true', help='No body pose')
@@ -79,8 +75,7 @@ input_vcodec = videoinfo["codec_name"]
 
 # define a writer object to write to a movidified file
 postfix = info["format"]["format_name"].split(",")[0]
-output_file = ".".join(video_file.split(".")[:-1])+".processed." + postfix
-
+output_file = ".".join(video_file.split(".")[:-1]) + ".processed." + postfix
 
 class Writer():
     def __init__(self, output_file, input_fps, input_framesize, input_pix_fmt,
@@ -92,7 +87,7 @@ class Writer():
             .input('pipe:',
                    format='rawvideo',
                    pix_fmt="bgr24",
-                   s='%sx%s'%(input_framesize[1],input_framesize[0]),
+                   s='%sx%s' % (input_framesize[1], input_framesize[0]),
                    r=input_fps)
             .output(output_file, pix_fmt=input_pix_fmt, vcodec=input_vcodec)
             .overwrite_output()
@@ -106,15 +101,14 @@ class Writer():
         self.ff_proc.stdin.close()
         self.ff_proc.wait()
 
-
 writer = None
-while(cap.isOpened()):
+while (cap.isOpened()):
     ret, frame = cap.read()
     if frame is None:
         break
 
     posed_frame = process_frame(frame, body=not args.no_body,
-                                       hands=not args.no_hands)
+                                hands=not args.no_hands)
 
     if writer is None:
         input_framesize = posed_frame.shape[:2]

backend/flask_api.py

@@ -1,18 +1,17 @@
+# from PIL import Image
+import base64
 import json
 import os
-
+import shutil
 from flask import Flask, request
 from flask_cors import *
 from flask_cors import CORS
 from flask_cors import cross_origin
-# from PIL import Image
-import base64
 
 from analyse_func import do_analysis
 from classify import api_dyna
 from db_project import get_status, process_db, lookup_data, add_new_data_dyna, add_new_data, create_table_dynamic, \
     create_table, create_db
-import shutil
 from demo_func import run_3d_for_video
 
 # str_res=""
@@ -52,7 +51,6 @@ result["leg_strength"]["level"]/["path"]
 result["standard"]["level"]/["path"]
 """
 
-
 # read txt file and change it to the list format
 def read_pose_processes_txt_result():
     file_dir = "pose_processed_txt"  # the file's path
@@ -146,14 +144,12 @@ def read_pose_processes_txt_result():
     global li_return
     li_return = dict_total[(str(global_index[0]) + ".txt")]
 
-
 def run_the_process(index: int = 0, height: float = 110):
     # sys.path.append("../analyse_func.py")
     # import analyse_func
     # print(global_index[0])
     do_analysis(global_index[0], height)
 
-
 # 把list转化为可以直接返回的json格式
 
 def list_to_json(li):
@@ -169,10 +165,8 @@ def list_to_json(li):
     return aJson
     # print(dic_data)
 
-
 app = Flask(__name__)
 
-
 def save_file(filed):
     base = "pose_source_images/"
     filename = filed.filename
@@ -200,7 +194,6 @@ def save_file(filed):
     print(filename)
     print(save_name)
 
-
 # 运行动态分析的入口
 def run_video(mode, num, ana=1):
     q = "./video/" + str(num) + ".mp4"
@@ -209,7 +202,6 @@ def run_video(mode, num, ana=1):
     process_db(num)
     return re
 
-
 def lookup_video(num, ana=0):
     q = "./video/" + str(num) + ".mp4"
     mode = lookup_data(num)
@@ -217,7 +209,6 @@ def lookup_video(num, ana=0):
     process_db(num)
     return re
 
-
 ###send a video and return a id marked the video (identical)
 @app.route('/get_videoid', methods=['POST'])
 @cross_origin()
@@ -238,7 +229,6 @@ def get_videoid():
 
     return {"id": str(global_video_index)}
 
-
 @app.route('/get_video_data', methods=['POST'])
 @cross_origin()
 def get_video_data():
@@ -250,7 +240,6 @@ def get_video_data():
     re = run_video(mode, global_video_index, 1)
     return re
 
-
 @app.route('/get_video_status', methods=['POST'])
 @cross_origin(supports_credentials=True)
 def get_video_status():
@@ -260,7 +249,6 @@ def get_video_status():
     # bool = get_status(global_video_index) == 1
     return {"state": get_status(global_video_index) == 1}
 
-
 @app.route('/query_video', methods=['POST'])
 @cross_origin(supports_credentials=True)
 def query_video():
@@ -270,16 +258,14 @@ def query_video():
     global_video_index = int(dict_video_["id"])
     if get_status(global_video_index) == 0:
         mode = str(lookup_data(global_video_index))  # get database mode
-        print("test_run_video"+mode + global_video_index)
+        print("test_run_video" + mode + global_video_index)
         return run_video(mode, global_video_index, 1)
     return lookup_video(global_video_index, 0)
 
-
 def get_3D(id):
     video_path = "video/" + id + ".mp4"
     run_3d_for_video(video_path)
 
-
 @app.route('/query_3D', methods=['POST'])
 @cross_origin()
 def query_3D():
@@ -295,8 +281,6 @@ def query_3D():
         # 'status':'finished'
     }
 
-
-
 # result={xx:xx,xx:xx}
 # result={power:{val:xx,path:xx},strength:{val:xx,path:xx}}
 
@@ -322,13 +306,11 @@ def send_form():
         "status": True if height is not None else False
     }
 
-
 def get_base64(path):
     with open(path, "rb") as f:
         base64_data = base64.b64encode(f.read())
         return base64_data.decode()
 
-
 @app.route('/get_result', methods=['POST'])
 @cross_origin()
 def run_project():
@@ -344,20 +326,19 @@ def run_project():
         "data": list_to_json(li_return),
     }
 
-
 if __name__ == '__main__':
     # read_pose_processes_txt_result()
     # print(list_to_json(li_return))
 
     # app.run()
     app.run(host="0.0.0.0", port=8090, debug=True)
-    #app.run(host="0.0.0.0", port=8090)
+    # app.run(host="0.0.0.0", port=8090)
     CORS(app, resouces=r'/*')
     # run_video("gaotaitui",19,1)
     # lookup_video(41, 0)
-    #get_3D("26")
-    #get_3D("39")
-    #get_3D("40")
-    #get_3D("23")
-    #get_3D("24")
-    #get_3D("25")
+    # get_3D("26")
+    # get_3D("39")
+    # get_3D("40")
+    # get_3D("23")
+    # get_3D("24")
+    # get_3D("25")

backend/gunicorn.conf.py

@@ -1,10 +1,10 @@
 workers = 5
 worker_class = "gevent"
 bind = "0.0.0.0:8090"
-#accesslog="/home/seecoder/Documents/pose-correction/pc-demo/log/gunicorn_access.log"
-#errorlog="/home/seecoder/Documents/pose-correction/pc-demo/log/gunicorn_error.log"
-#pidfile = '/home/seecoder/Documents/pose-correction/pc-demo/run/gunicorn.pid'
-accesslog="/root/openpose/pose-correction/pc-demo/log/gunicorn_access.log"
-errorlog="/root/openpose/pose-correction/pc-demo/log/gunicorn_error.log"
+# accesslog="/home/seecoder/Documents/pose-correction/pc-demo/log/gunicorn_access.log"
+# errorlog="/home/seecoder/Documents/pose-correction/pc-demo/log/gunicorn_error.log"
+# pidfile = '/home/seecoder/Documents/pose-correction/pc-demo/run/gunicorn.pid'
+accesslog = "/root/openpose/pose-correction/pc-demo/log/gunicorn_access.log"
+errorlog = "/root/openpose/pose-correction/pc-demo/log/gunicorn_error.log"
 pidfile = '/root/openpose/pose-correction/pc-demo/run/gunicorn.pid'
 capture_output = True

backend/main.py

@@ -1,7 +1,10 @@
 from flask import Flask
+
 app = Flask(__name__)
+
 @app.route("/")
 def hello():
     return "<h1>Hello There!</h1>"
+
 if __name__ == "__main__":
-    app.run(host='0.0.0.0')
+    app.run(host='0.0.0.0')

backend/src/body.py

@@ -1,9 +1,9 @@
 import cv2
-import numpy as np
 import math
-from scipy.ndimage.filters import gaussian_filter
 import matplotlib.pyplot as plt
+import numpy as np
 import torch
+from scipy.ndimage.filters import gaussian_filter
 
 from src import util
 from src.model import bodypose_model
@@ -78,7 +78,8 @@ class Body(object):
             map_down[:, :-1] = one_heatmap[:, 1:]
 
             peaks_binary = np.logical_and.reduce(
-                (one_heatmap >= map_left, one_heatmap >= map_right, one_heatmap >= map_up, one_heatmap >= map_down, one_heatmap > thre1))
+                (one_heatmap >= map_left, one_heatmap >= map_right, one_heatmap >= map_up, one_heatmap >= map_down,
+                 one_heatmap > thre1))
             peaks = list(zip(np.nonzero(peaks_binary)[1], np.nonzero(peaks_binary)[0]))  # note reverse
             peaks_with_score = [x + (map_ori[x[1], x[0]],) for x in peaks]
             peak_id = range(peak_counter, peak_counter + len(peaks))
@@ -208,7 +209,7 @@ if __name__ == "__main__":
     body_estimation = Body('../model/body_pose_model.pth')
 
     test_image = '../images/ski.jpg'
-    #test_image='.。/capture_image/capture_image3.png'
+    # test_image='.。/capture_image/capture_image3.png'
     oriImg = cv2.imread(test_image)  # B,G,R order
     candidate, subset = body_estimation(oriImg)
     canvas = util.draw_bodypose(oriImg, candidate, subset)

backend/src/hand.py

@@ -1,11 +1,11 @@
 import cv2
 import numpy as np
-from scipy.ndimage.filters import gaussian_filter
 import torch
+from scipy.ndimage.filters import gaussian_filter
 from skimage.measure import label
 
-from src.model import handpose_model
 from src import util
+from src.model import handpose_model
 
 class Hand(object):
     def __init__(self, model_path):

backend/src/hand_model_output_size.json

@@ -1,992 +1,992 @@
 {
-    "10":1,
-    "11":1,
-    "12":1,
-    "13":1,
-    "14":1,
-    "15":1,
-    "16":2,
-    "17":2,
-    "18":2,
-    "19":2,
-    "20":2,
-    "21":2,
-    "22":2,
-    "23":2,
-    "24":3,
-    "25":3,
-    "26":3,
-    "27":3,
-    "28":3,
-    "29":3,
-    "30":3,
-    "31":3,
-    "32":4,
-    "33":4,
-    "34":4,
-    "35":4,
-    "36":4,
-    "37":4,
-    "38":4,
-    "39":4,
-    "40":5,
-    "41":5,
-    "42":5,
-    "43":5,
-    "44":5,
-    "45":5,
-    "46":5,
-    "47":5,
-    "48":6,
-    "49":6,
-    "50":6,
-    "51":6,
-    "52":6,
-    "53":6,
-    "54":6,
-    "55":6,
-    "56":7,
-    "57":7,
-    "58":7,
-    "59":7,
-    "60":7,
-    "61":7,
-    "62":7,
-    "63":7,
-    "64":8,
-    "65":8,
-    "66":8,
-    "67":8,
-    "68":8,
-    "69":8,
-    "70":8,
-    "71":8,
-    "72":9,
-    "73":9,
-    "74":9,
-    "75":9,
-    "76":9,
-    "77":9,
-    "78":9,
-    "79":9,
-    "80":10,
-    "81":10,
-    "82":10,
-    "83":10,
-    "84":10,
-    "85":10,
-    "86":10,
-    "87":10,
-    "88":11,
-    "89":11,
-    "90":11,
-    "91":11,
-    "92":11,
-    "93":11,
-    "94":11,
-    "95":11,
-    "96":12,
-    "97":12,
-    "98":12,
-    "99":12,
-    "100":12,
-    "101":12,
-    "102":12,
-    "103":12,
-    "104":13,
-    "105":13,
-    "106":13,
-    "107":13,
-    "108":13,
-    "109":13,
-    "110":13,
-    "111":13,
-    "112":14,
-    "113":14,
-    "114":14,
-    "115":14,
-    "116":14,
-    "117":14,
-    "118":14,
-    "119":14,
-    "120":15,
-    "121":15,
-    "122":15,
-    "123":15,
-    "124":15,
-    "125":15,
-    "126":15,
-    "127":15,
-    "128":16,
-    "129":16,
-    "130":16,
-    "131":16,
-    "132":16,
-    "133":16,
-    "134":16,
-    "135":16,
-    "136":17,
-    "137":17,
-    "138":17,
-    "139":17,
-    "140":17,
-    "141":17,
-    "142":17,
-    "143":17,
-    "144":18,
-    "145":18,
-    "146":18,
-    "147":18,
-    "148":18,
-    "149":18,
-    "150":18,
-    "151":18,
-    "152":19,
-    "153":19,
-    "154":19,
-    "155":19,
-    "156":19,
-    "157":19,
-    "158":19,
-    "159":19,
-    "160":20,
-    "161":20,
-    "162":20,
-    "163":20,
-    "164":20,
-    "165":20,
-    "166":20,
-    "167":20,
-    "168":21,
-    "169":21,
-    "170":21,
-    "171":21,
-    "172":21,
-    "173":21,
-    "174":21,
-    "175":21,
-    "176":22,
-    "177":22,
-    "178":22,
-    "179":22,
-    "180":22,
-    "181":22,
-    "182":22,
-    "183":22,
-    "184":23,
-    "185":23,
-    "186":23,
-    "187":23,
-    "188":23,
-    "189":23,
-    "190":23,
-    "191":23,
-    "192":24,
-    "193":24,
-    "194":24,
-    "195":24,
-    "196":24,
-    "197":24,
-    "198":24,
-    "199":24,
-    "200":25,
-    "201":25,
-    "202":25,
-    "203":25,
-    "204":25,
-    "205":25,
-    "206":25,
-    "207":25,
-    "208":26,
-    "209":26,
-    "210":26,
-    "211":26,
-    "212":26,
-    "213":26,
-    "214":26,
-    "215":26,
-    "216":27,
-    "217":27,
-    "218":27,
-    "219":27,
-    "220":27,
-    "221":27,
-    "222":27,
-    "223":27,
-    "224":28,
-    "225":28,
-    "226":28,
-    "227":28,
-    "228":28,
-    "229":28,
-    "230":28,
-    "231":28,
-    "232":29,
-    "233":29,
-    "234":29,
-    "235":29,
-    "236":29,
-    "237":29,
-    "238":29,
-    "239":29,
-    "240":30,
-    "241":30,
-    "242":30,
-    "243":30,
-    "244":30,
-    "245":30,
-    "246":30,
-    "247":30,
-    "248":31,
-    "249":31,
-    "250":31,
-    "251":31,
-    "252":31,
-    "253":31,
-    "254":31,
-    "255":31,
-    "256":32,
-    "257":32,
-    "258":32,
-    "259":32,
-    "260":32,
-    "261":32,
-    "262":32,
-    "263":32,
-    "264":33,
-    "265":33,
-    "266":33,
-    "267":33,
-    "268":33,
-    "269":33,
-    "270":33,
-    "271":33,
-    "272":34,
-    "273":34,
-    "274":34,
-    "275":34,
-    "276":34,
-    "277":34,
-    "278":34,
-    "279":34,
-    "280":35,
-    "281":35,
-    "282":35,
-    "283":35,
-    "284":35,
-    "285":35,
-    "286":35,
-    "287":35,
-    "288":36,
-    "289":36,
-    "290":36,
-    "291":36,
-    "292":36,
-    "293":36,
-    "294":36,
-    "295":36,
-    "296":37,
-    "297":37,
-    "298":37,
-    "299":37,
-    "300":37,
-    "301":37,
-    "302":37,
-    "303":37,
-    "304":38,
-    "305":38,
-    "306":38,
-    "307":38,
-    "308":38,
-    "309":38,
-    "310":38,
-    "311":38,
-    "312":39,
-    "313":39,
-    "314":39,
-    "315":39,
-    "316":39,
-    "317":39,
-    "318":39,
-    "319":39,
-    "320":40,
-    "321":40,
-    "322":40,
-    "323":40,
-    "324":40,
-    "325":40,
-    "326":40,
-    "327":40,
-    "328":41,
-    "329":41,
-    "330":41,
-    "331":41,
-    "332":41,
-    "333":41,
-    "334":41,
-    "335":41,
-    "336":42,
-    "337":42,
-    "338":42,
-    "339":42,
-    "340":42,
-    "341":42,
-    "342":42,
-    "343":42,
-    "344":43,
-    "345":43,
-    "346":43,
-    "347":43,
-    "348":43,
-    "349":43,
-    "350":43,
-    "351":43,
-    "352":44,
-    "353":44,
-    "354":44,
-    "355":44,
-    "356":44,
-    "357":44,
-    "358":44,
-    "359":44,
-    "360":45,
-    "361":45,
-    "362":45,
-    "363":45,
-    "364":45,
-    "365":45,
-    "366":45,
-    "367":45,
-    "368":46,
-    "369":46,
-    "370":46,
-    "371":46,
-    "372":46,
-    "373":46,
-    "374":46,
-    "375":46,
-    "376":47,
-    "377":47,
-    "378":47,
-    "379":47,
-    "380":47,
-    "381":47,
-    "382":47,
-    "383":47,
-    "384":48,
-    "385":48,
-    "386":48,
-    "387":48,
-    "388":48,
-    "389":48,
-    "390":48,
-    "391":48,
-    "392":49,
-    "393":49,
-    "394":49,
-    "395":49,
-    "396":49,
-    "397":49,
-    "398":49,
-    "399":49,
-    "400":50,
-    "401":50,
-    "402":50,
-    "403":50,
-    "404":50,
-    "405":50,
-    "406":50,
-    "407":50,
-    "408":51,
-    "409":51,
-    "410":51,
-    "411":51,
-    "412":51,
-    "413":51,
-    "414":51,
-    "415":51,
-    "416":52,
-    "417":52,
-    "418":52,
-    "419":52,
-    "420":52,
-    "421":52,
-    "422":52,
-    "423":52,
-    "424":53,
-    "425":53,
-    "426":53,
-    "427":53,
-    "428":53,
-    "429":53,
-    "430":53,
-    "431":53,
-    "432":54,
-    "433":54,
-    "434":54,
-    "435":54,
-    "436":54,
-    "437":54,
-    "438":54,
-    "439":54,
-    "440":55,
-    "441":55,
-    "442":55,
-    "443":55,
-    "444":55,
-    "445":55,
-    "446":55,
-    "447":55,
-    "448":56,
-    "449":56,
-    "450":56,
-    "451":56,
-    "452":56,
-    "453":56,
-    "454":56,
-    "455":56,
-    "456":57,
-    "457":57,
-    "458":57,
-    "459":57,
-    "460":57,
-    "461":57,
-    "462":57,
-    "463":57,
-    "464":58,
-    "465":58,
-    "466":58,
-    "467":58,
-    "468":58,
-    "469":58,
-    "470":58,
-    "471":58,
-    "472":59,
-    "473":59,
-    "474":59,
-    "475":59,
-    "476":59,
-    "477":59,
-    "478":59,
-    "479":59,
-    "480":60,
-    "481":60,
-    "482":60,
-    "483":60,
-    "484":60,
-    "485":60,
-    "486":60,
-    "487":60,
-    "488":61,
-    "489":61,
-    "490":61,
-    "491":61,
-    "492":61,
-    "493":61,
-    "494":61,
-    "495":61,
-    "496":62,
-    "497":62,
-    "498":62,
-    "499":62,
-    "500":62,
-    "501":62,
-    "502":62,
-    "503":62,
-    "504":63,
-    "505":63,
-    "506":63,
-    "507":63,
-    "508":63,
-    "509":63,
-    "510":63,
-    "511":63,
-    "512":64,
-    "513":64,
-    "514":64,
-    "515":64,
-    "516":64,
-    "517":64,
-    "518":64,
-    "519":64,
-    "520":65,
-    "521":65,
-    "522":65,
-    "523":65,
-    "524":65,
-    "525":65,
-    "526":65,
-    "527":65,
-    "528":66,
-    "529":66,
-    "530":66,
-    "531":66,
-    "532":66,
-    "533":66,
-    "534":66,
-    "535":66,
-    "536":67,
-    "537":67,
-    "538":67,
-    "539":67,
-    "540":67,
-    "541":67,
-    "542":67,
-    "543":67,
-    "544":68,
-    "545":68,
-    "546":68,
-    "547":68,
-    "548":68,
-    "549":68,
-    "550":68,
-    "551":68,
-    "552":69,
-    "553":69,
-    "554":69,
-    "555":69,
-    "556":69,
-    "557":69,
-    "558":69,
-    "559":69,
-    "560":70,
-    "561":70,
-    "562":70,
-    "563":70,
-    "564":70,
-    "565":70,
-    "566":70,
-    "567":70,
-    "568":71,
-    "569":71,
-    "570":71,
-    "571":71,
-    "572":71,
-    "573":71,
-    "574":71,
-    "575":71,
-    "576":72,
-    "577":72,
-    "578":72,
-    "579":72,
-    "580":72,
-    "581":72,
-    "582":72,
-    "583":72,
-    "584":73,
-    "585":73,
-    "586":73,
-    "587":73,
-    "588":73,
-    "589":73,
-    "590":73,
-    "591":73,
-    "592":74,
-    "593":74,
-    "594":74,
-    "595":74,
-    "596":74,
-    "597":74,
-    "598":74,
-    "599":74,
-    "600":75,
-    "601":75,
-    "602":75,
-    "603":75,
-    "604":75,
-    "605":75,
-    "606":75,
-    "607":75,
-    "608":76,
-    "609":76,
-    "610":76,
-    "611":76,
-    "612":76,
-    "613":76,
-    "614":76,
-    "615":76,
-    "616":77,
-    "617":77,
-    "618":77,
-    "619":77,
-    "620":77,
-    "621":77,
-    "622":77,
-    "623":77,
-    "624":78,
-    "625":78,
-    "626":78,
-    "627":78,
-    "628":78,
-    "629":78,
-    "630":78,
-    "631":78,
-    "632":79,
-    "633":79,
-    "634":79,
-    "635":79,
-    "636":79,
-    "637":79,
-    "638":79,
-    "639":79,
-    "640":80,
-    "641":80,
-    "642":80,
-    "643":80,
-    "644":80,
-    "645":80,
-    "646":80,
-    "647":80,
-    "648":81,
-    "649":81,
-    "650":81,
-    "651":81,
-    "652":81,
-    "653":81,
-    "654":81,
-    "655":81,
-    "656":82,
-    "657":82,
-    "658":82,
-    "659":82,
-    "660":82,
-    "661":82,
-    "662":82,
-    "663":82,
-    "664":83,
-    "665":83,
-    "666":83,
-    "667":83,
-    "668":83,
-    "669":83,
-    "670":83,
-    "671":83,
-    "672":84,
-    "673":84,
-    "674":84,
-    "675":84,
-    "676":84,
-    "677":84,
-    "678":84,
-    "679":84,
-    "680":85,
-    "681":85,
-    "682":85,
-    "683":85,
-    "684":85,
-    "685":85,
-    "686":85,
-    "687":85,
-    "688":86,
-    "689":86,
-    "690":86,
-    "691":86,
-    "692":86,
-    "693":86,
-    "694":86,
-    "695":86,
-    "696":87,
-    "697":87,
-    "698":87,
-    "699":87,
-    "700":87,
-    "701":87,
-    "702":87,
-    "703":87,
-    "704":88,
-    "705":88,
-    "706":88,
-    "707":88,
-    "708":88,
-    "709":88,
-    "710":88,
-    "711":88,
-    "712":89,
-    "713":89,
-    "714":89,
-    "715":89,
-    "716":89,
-    "717":89,
-    "718":89,
-    "719":89,
-    "720":90,
-    "721":90,
-    "722":90,
-    "723":90,
-    "724":90,
-    "725":90,
-    "726":90,
-    "727":90,
-    "728":91,
-    "729":91,
-    "730":91,
-    "731":91,
-    "732":91,
-    "733":91,
-    "734":91,
-    "735":91,
-    "736":92,
-    "737":92,
-    "738":92,
-    "739":92,
-    "740":92,
-    "741":92,
-    "742":92,
-    "743":92,
-    "744":93,
-    "745":93,
-    "746":93,
-    "747":93,
-    "748":93,
-    "749":93,
-    "750":93,
-    "751":93,
-    "752":94,
-    "753":94,
-    "754":94,
-    "755":94,
-    "756":94,
-    "757":94,
-    "758":94,
-    "759":94,
-    "760":95,
-    "761":95,
-    "762":95,
-    "763":95,
-    "764":95,
-    "765":95,
-    "766":95,
-    "767":95,
-    "768":96,
-    "769":96,
-    "770":96,
-    "771":96,
-    "772":96,
-    "773":96,
-    "774":96,
-    "775":96,
-    "776":97,
-    "777":97,
-    "778":97,
-    "779":97,
-    "780":97,
-    "781":97,
-    "782":97,
-    "783":97,
-    "784":98,
-    "785":98,
-    "786":98,
-    "787":98,
-    "788":98,
-    "789":98,
-    "790":98,
-    "791":98,
-    "792":99,
-    "793":99,
-    "794":99,
-    "795":99,
-    "796":99,
-    "797":99,
-    "798":99,
-    "799":99,
-    "800":100,
-    "801":100,
-    "802":100,
-    "803":100,
-    "804":100,
-    "805":100,
-    "806":100,
-    "807":100,
-    "808":101,
-    "809":101,
-    "810":101,
-    "811":101,
-    "812":101,
-    "813":101,
-    "814":101,
-    "815":101,
-    "816":102,
-    "817":102,
-    "818":102,
-    "819":102,
-    "820":102,
-    "821":102,
-    "822":102,
-    "823":102,
-    "824":103,
-    "825":103,
-    "826":103,
-    "827":103,
-    "828":103,
-    "829":103,
-    "830":103,
-    "831":103,
-    "832":104,
-    "833":104,
-    "834":104,
-    "835":104,
-    "836":104,
-    "837":104,
-    "838":104,
-    "839":104,
-    "840":105,
-    "841":105,
-    "842":105,
-    "843":105,
-    "844":105,
-    "845":105,
-    "846":105,
-    "847":105,
-    "848":106,
-    "849":106,
-    "850":106,
-    "851":106,
-    "852":106,
-    "853":106,
-    "854":106,
-    "855":106,
-    "856":107,
-    "857":107,
-    "858":107,
-    "859":107,
-    "860":107,
-    "861":107,
-    "862":107,
-    "863":107,
-    "864":108,
-    "865":108,
-    "866":108,
-    "867":108,
-    "868":108,
-    "869":108,
-    "870":108,
-    "871":108,
-    "872":109,
-    "873":109,
-    "874":109,
-    "875":109,
-    "876":109,
-    "877":109,
-    "878":109,
-    "879":109,
-    "880":110,
-    "881":110,
-    "882":110,
-    "883":110,
-    "884":110,
-    "885":110,
-    "886":110,
-    "887":110,
-    "888":111,
-    "889":111,
-    "890":111,
-    "891":111,
-    "892":111,
-    "893":111,
-    "894":111,
-    "895":111,
-    "896":112,
-    "897":112,
-    "898":112,
-    "899":112,
-    "900":112,
-    "901":112,
-    "902":112,
-    "903":112,
-    "904":113,
-    "905":113,
-    "906":113,
-    "907":113,
-    "908":113,
-    "909":113,
-    "910":113,
-    "911":113,
-    "912":114,
-    "913":114,
-    "914":114,
-    "915":114,
-    "916":114,
-    "917":114,
-    "918":114,
-    "919":114,
-    "920":115,
-    "921":115,
-    "922":115,
-    "923":115,
-    "924":115,
-    "925":115,
-    "926":115,
-    "927":115,
-    "928":116,
-    "929":116,
-    "930":116,
-    "931":116,
-    "932":116,
-    "933":116,
-    "934":116,
-    "935":116,
-    "936":117,
-    "937":117,
-    "938":117,
-    "939":117,
-    "940":117,
-    "941":117,
-    "942":117,
-    "943":117,
-    "944":118,
-    "945":118,
-    "946":118,
-    "947":118,
-    "948":118,
-    "949":118,
-    "950":118,
-    "951":118,
-    "952":119,
-    "953":119,
-    "954":119,
-    "955":119,
-    "956":119,
-    "957":119,
-    "958":119,
-    "959":119,
-    "960":120,
-    "961":120,
-    "962":120,
-    "963":120,
-    "964":120,
-    "965":120,
-    "966":120,
-    "967":120,
-    "968":121,
-    "969":121,
-    "970":121,
-    "971":121,
-    "972":121,
-    "973":121,
-    "974":121,
-    "975":121,
-    "976":122,
-    "977":122,
-    "978":122,
-    "979":122,
-    "980":122,
-    "981":122,
-    "982":122,
-    "983":122,
-    "984":123,
-    "985":123,
-    "986":123,
-    "987":123,
-    "988":123,
-    "989":123,
-    "990":123,
-    "991":123,
-    "992":124,
-    "993":124,
-    "994":124,
-    "995":124,
-    "996":124,
-    "997":124,
-    "998":124,
-    "999":124
+  "10": 1,
+  "11": 1,
+  "12": 1,
+  "13": 1,
+  "14": 1,
+  "15": 1,
+  "16": 2,
+  "17": 2,
+  "18": 2,
+  "19": 2,
+  "20": 2,
+  "21": 2,
+  "22": 2,
+  "23": 2,
+  "24": 3,
+  "25": 3,
+  "26": 3,
+  "27": 3,
+  "28": 3,
+  "29": 3,
+  "30": 3,
+  "31": 3,
+  "32": 4,
+  "33": 4,
+  "34": 4,
+  "35": 4,
+  "36": 4,
+  "37": 4,
+  "38": 4,
+  "39": 4,
+  "40": 5,
+  "41": 5,
+  "42": 5,
+  "43": 5,
+  "44": 5,
+  "45": 5,
+  "46": 5,
+  "47": 5,
+  "48": 6,
+  "49": 6,
+  "50": 6,
+  "51": 6,
+  "52": 6,
+  "53": 6,
+  "54": 6,
+  "55": 6,
+  "56": 7,
+  "57": 7,
+  "58": 7,
+  "59": 7,
+  "60": 7,
+  "61": 7,
+  "62": 7,
+  "63": 7,
+  "64": 8,
+  "65": 8,
+  "66": 8,
+  "67": 8,
+  "68": 8,
+  "69": 8,
+  "70": 8,
+  "71": 8,
+  "72": 9,
+  "73": 9,
+  "74": 9,
+  "75": 9,
+  "76": 9,
+  "77": 9,
+  "78": 9,
+  "79": 9,
+  "80": 10,
+  "81": 10,
+  "82": 10,
+  "83": 10,
+  "84": 10,
+  "85": 10,
+  "86": 10,
+  "87": 10,
+  "88": 11,
+  "89": 11,
+  "90": 11,
+  "91": 11,
+  "92": 11,
+  "93": 11,
+  "94": 11,
+  "95": 11,
+  "96": 12,
+  "97": 12,
+  "98": 12,
+  "99": 12,
+  "100": 12,
+  "101": 12,
+  "102": 12,
+  "103": 12,
+  "104": 13,
+  "105": 13,
+  "106": 13,
+  "107": 13,
+  "108": 13,
+  "109": 13,
+  "110": 13,
+  "111": 13,
+  "112": 14,
+  "113": 14,
+  "114": 14,
+  "115": 14,
+  "116": 14,
+  "117": 14,
+  "118": 14,
+  "119": 14,
+  "120": 15,
+  "121": 15,
+  "122": 15,
+  "123": 15,
+  "124": 15,
+  "125": 15,
+  "126": 15,
+  "127": 15,
+  "128": 16,
+  "129": 16,
+  "130": 16,
+  "131": 16,
+  "132": 16,
+  "133": 16,
+  "134": 16,
+  "135": 16,
+  "136": 17,
+  "137": 17,
+  "138": 17,
+  "139": 17,
+  "140": 17,
+  "141": 17,
+  "142": 17,
+  "143": 17,
+  "144": 18,
+  "145": 18,
+  "146": 18,
+  "147": 18,
+  "148": 18,
+  "149": 18,
+  "150": 18,
+  "151": 18,
+  "152": 19,
+  "153": 19,
+  "154": 19,
+  "155": 19,
+  "156": 19,
+  "157": 19,
+  "158": 19,
+  "159": 19,
+  "160": 20,
+  "161": 20,
+  "162": 20,
+  "163": 20,
+  "164": 20,
+  "165": 20,
+  "166": 20,
+  "167": 20,
+  "168": 21,
+  "169": 21,
+  "170": 21,
+  "171": 21,
+  "172": 21,
+  "173": 21,
+  "174": 21,
+  "175": 21,
+  "176": 22,
+  "177": 22,
+  "178": 22,
+  "179": 22,
+  "180": 22,
+  "181": 22,
+  "182": 22,
+  "183": 22,
+  "184": 23,
+  "185": 23,
+  "186": 23,
+  "187": 23,
+  "188": 23,
+  "189": 23,
+  "190": 23,
+  "191": 23,
+  "192": 24,
+  "193": 24,
+  "194": 24,
+  "195": 24,
+  "196": 24,
+  "197": 24,
+  "198": 24,
+  "199": 24,
+  "200": 25,
+  "201": 25,
+  "202": 25,
+  "203": 25,
+  "204": 25,
+  "205": 25,
+  "206": 25,
+  "207": 25,
+  "208": 26,
+  "209": 26,
+  "210": 26,
+  "211": 26,
+  "212": 26,
+  "213": 26,
+  "214": 26,
+  "215": 26,
+  "216": 27,
+  "217": 27,
+  "218": 27,
+  "219": 27,
+  "220": 27,
+  "221": 27,
+  "222": 27,
+  "223": 27,
+  "224": 28,
+  "225": 28,
+  "226": 28,
+  "227": 28,
+  "228": 28,
+  "229": 28,
+  "230": 28,
+  "231": 28,
+  "232": 29,
+  "233": 29,
+  "234": 29,
+  "235": 29,
+  "236": 29,
+  "237": 29,
+  "238": 29,
+  "239": 29,
+  "240": 30,
+  "241": 30,
+  "242": 30,
+  "243": 30,
+  "244": 30,
+  "245": 30,
+  "246": 30,
+  "247": 30,
+  "248": 31,
+  "249": 31,
+  "250": 31,
+  "251": 31,
+  "252": 31,
+  "253": 31,
+  "254": 31,
+  "255": 31,
+  "256": 32,
+  "257": 32,
+  "258": 32,
+  "259": 32,
+  "260": 32,
+  "261": 32,
+  "262": 32,
+  "263": 32,
+  "264": 33,
+  "265": 33,
+  "266": 33,
+  "267": 33,
+  "268": 33,
+  "269": 33,
+  "270": 33,
+  "271": 33,
+  "272": 34,
+  "273": 34,
+  "274": 34,
+  "275": 34,
+  "276": 34,
+  "277": 34,
+  "278": 34,
+  "279": 34,
+  "280": 35,
+  "281": 35,
+  "282": 35,
+  "283": 35,
+  "284": 35,
+  "285": 35,
+  "286": 35,
+  "287": 35,
+  "288": 36,
+  "289": 36,
+  "290": 36,
+  "291": 36,
+  "292": 36,
+  "293": 36,
+  "294": 36,
+  "295": 36,
+  "296": 37,
+  "297": 37,
+  "298": 37,
+  "299": 37,
+  "300": 37,
+  "301": 37,
+  "302": 37,
+  "303": 37,
+  "304": 38,
+  "305": 38,
+  "306": 38,
+  "307": 38,
+  "308": 38,
+  "309": 38,
+  "310": 38,
+  "311": 38,
+  "312": 39,
+  "313": 39,
+  "314": 39,
+  "315": 39,
+  "316": 39,
+  "317": 39,
+  "318": 39,
+  "319": 39,
+  "320": 40,
+  "321": 40,
+  "322": 40,
+  "323": 40,
+  "324": 40,
+  "325": 40,
+  "326": 40,
+  "327": 40,
+  "328": 41,
+  "329": 41,
+  "330": 41,
+  "331": 41,
+  "332": 41,
+  "333": 41,
+  "334": 41,
+  "335": 41,
+  "336": 42,
+  "337": 42,
+  "338": 42,
+  "339": 42,
+  "340": 42,
+  "341": 42,
+  "342": 42,
+  "343": 42,
+  "344": 43,
+  "345": 43,
+  "346": 43,
+  "347": 43,
+  "348": 43,
+  "349": 43,
+  "350": 43,
+  "351": 43,
+  "352": 44,
+  "353": 44,
+  "354": 44,
+  "355": 44,
+  "356": 44,
+  "357": 44,
+  "358": 44,
+  "359": 44,
+  "360": 45,
+  "361": 45,
+  "362": 45,
+  "363": 45,
+  "364": 45,
+  "365": 45,
+  "366": 45,
+  "367": 45,
+  "368": 46,
+  "369": 46,
+  "370": 46,
+  "371": 46,
+  "372": 46,
+  "373": 46,
+  "374": 46,
+  "375": 46,
+  "376": 47,
+  "377": 47,
+  "378": 47,
+  "379": 47,
+  "380": 47,
+  "381": 47,
+  "382": 47,
+  "383": 47,
+  "384": 48,
+  "385": 48,
+  "386": 48,
+  "387": 48,
+  "388": 48,
+  "389": 48,
+  "390": 48,
+  "391": 48,
+  "392": 49,
+  "393": 49,
+  "394": 49,
+  "395": 49,
+  "396": 49,
+  "397": 49,
+  "398": 49,
+  "399": 49,
+  "400": 50,
+  "401": 50,
+  "402": 50,
+  "403": 50,
+  "404": 50,
+  "405": 50,
+  "406": 50,
+  "407": 50,
+  "408": 51,
+  "409": 51,
+  "410": 51,
+  "411": 51,
+  "412": 51,
+  "413": 51,
+  "414": 51,
+  "415": 51,
+  "416": 52,
+  "417": 52,
+  "418": 52,
+  "419": 52,
+  "420": 52,
+  "421": 52,
+  "422": 52,
+  "423": 52,
+  "424": 53,
+  "425": 53,
+  "426": 53,
+  "427": 53,
+  "428": 53,
+  "429": 53,
+  "430": 53,
+  "431": 53,
+  "432": 54,
+  "433": 54,
+  "434": 54,
+  "435": 54,
+  "436": 54,
+  "437": 54,
+  "438": 54,
+  "439": 54,
+  "440": 55,
+  "441": 55,
+  "442": 55,
+  "443": 55,
+  "444": 55,
+  "445": 55,
+  "446": 55,
+  "447": 55,
+  "448": 56,
+  "449": 56,
+  "450": 56,
+  "451": 56,
+  "452": 56,
+  "453": 56,
+  "454": 56,
+  "455": 56,
+  "456": 57,
+  "457": 57,
+  "458": 57,
+  "459": 57,
+  "460": 57,
+  "461": 57,
+  "462": 57,
+  "463": 57,
+  "464": 58,
+  "465": 58,
+  "466": 58,
+  "467": 58,
+  "468": 58,
+  "469": 58,
+  "470": 58,
+  "471": 58,
+  "472": 59,
+  "473": 59,
+  "474": 59,
+  "475": 59,
+  "476": 59,
+  "477": 59,
+  "478": 59,
+  "479": 59,
+  "480": 60,
+  "481": 60,
+  "482": 60,
+  "483": 60,
+  "484": 60,
+  "485": 60,
+  "486": 60,
+  "487": 60,
+  "488": 61,
+  "489": 61,
+  "490": 61,
+  "491": 61,
+  "492": 61,
+  "493": 61,
+  "494": 61,
+  "495": 61,
+  "496": 62,
+  "497": 62,
+  "498": 62,
+  "499": 62,
+  "500": 62,
+  "501": 62,
+  "502": 62,
+  "503": 62,
+  "504": 63,
+  "505": 63,
+  "506": 63,
+  "507": 63,
+  "508": 63,
+  "509": 63,
+  "510": 63,
+  "511": 63,
+  "512": 64,
+  "513": 64,
+  "514": 64,
+  "515": 64,
+  "516": 64,
+  "517": 64,
+  "518": 64,
+  "519": 64,
+  "520": 65,
+  "521": 65,
+  "522": 65,
+  "523": 65,
+  "524": 65,
+  "525": 65,
+  "526": 65,
+  "527": 65,
+  "528": 66,
+  "529": 66,
+  "530": 66,
+  "531": 66,
+  "532": 66,
+  "533": 66,
+  "534": 66,
+  "535": 66,
+  "536": 67,
+  "537": 67,
+  "538": 67,
+  "539": 67,
+  "540": 67,
+  "541": 67,
+  "542": 67,
+  "543": 67,
+  "544": 68,
+  "545": 68,
+  "546": 68,
+  "547": 68,
+  "548": 68,
+  "549": 68,
+  "550": 68,
+  "551": 68,
+  "552": 69,
+  "553": 69,
+  "554": 69,
+  "555": 69,
+  "556": 69,
+  "557": 69,
+  "558": 69,
+  "559": 69,
+  "560": 70,
+  "561": 70,
+  "562": 70,
+  "563": 70,
+  "564": 70,
+  "565": 70,
+  "566": 70,
+  "567": 70,
+  "568": 71,
+  "569": 71,
+  "570": 71,
+  "571": 71,
+  "572": 71,
+  "573": 71,
+  "574": 71,
+  "575": 71,
+  "576": 72,
+  "577": 72,
+  "578": 72,
+  "579": 72,
+  "580": 72,
+  "581": 72,
+  "582": 72,
+  "583": 72,
+  "584": 73,
+  "585": 73,
+  "586": 73,
+  "587": 73,
+  "588": 73,
+  "589": 73,
+  "590": 73,
+  "591": 73,
+  "592": 74,
+  "593": 74,
+  "594": 74,
+  "595": 74,
+  "596": 74,
+  "597": 74,
+  "598": 74,
+  "599": 74,
+  "600": 75,
+  "601": 75,
+  "602": 75,
+  "603": 75,
+  "604": 75,
+  "605": 75,
+  "606": 75,
+  "607": 75,
+  "608": 76,
+  "609": 76,
+  "610": 76,
+  "611": 76,
+  "612": 76,
+  "613": 76,
+  "614": 76,
+  "615": 76,
+  "616": 77,
+  "617": 77,
+  "618": 77,
+  "619": 77,
+  "620": 77,
+  "621": 77,
+  "622": 77,
+  "623": 77,
+  "624": 78,
+  "625": 78,
+  "626": 78,
+  "627": 78,
+  "628": 78,
+  "629": 78,
+  "630": 78,
+  "631": 78,
+  "632": 79,
+  "633": 79,
+  "634": 79,
+  "635": 79,
+  "636": 79,
+  "637": 79,
+  "638": 79,
+  "639": 79,
+  "640": 80,
+  "641": 80,
+  "642": 80,
+  "643": 80,
+  "644": 80,
+  "645": 80,
+  "646": 80,
+  "647": 80,
+  "648": 81,
+  "649": 81,
+  "650": 81,
+  "651": 81,
+  "652": 81,
+  "653": 81,
+  "654": 81,
+  "655": 81,
+  "656": 82,
+  "657": 82,
+  "658": 82,
+  "659": 82,
+  "660": 82,
+  "661": 82,
+  "662": 82,
+  "663": 82,
+  "664": 83,
+  "665": 83,
+  "666": 83,
+  "667": 83,
+  "668": 83,
+  "669": 83,
+  "670": 83,
+  "671": 83,
+  "672": 84,
+  "673": 84,
+  "674": 84,
+  "675": 84,
+  "676": 84,
+  "677": 84,
+  "678": 84,
+  "679": 84,
+  "680": 85,
+  "681": 85,
+  "682": 85,
+  "683": 85,
+  "684": 85,
+  "685": 85,
+  "686": 85,
+  "687": 85,
+  "688": 86,
+  "689": 86,
+  "690": 86,
+  "691": 86,
+  "692": 86,
+  "693": 86,
+  "694": 86,
+  "695": 86,
+  "696": 87,
+  "697": 87,
+  "698": 87,
+  "699": 87,
+  "700": 87,
+  "701": 87,
+  "702": 87,
+  "703": 87,
+  "704": 88,
+  "705": 88,
+  "706": 88,
+  "707": 88,
+  "708": 88,
+  "709": 88,
+  "710": 88,
+  "711": 88,
+  "712": 89,
+  "713": 89,
+  "714": 89,
+  "715": 89,
+  "716": 89,
+  "717": 89,
+  "718": 89,
+  "719": 89,
+  "720": 90,
+  "721": 90,
+  "722": 90,
+  "723": 90,
+  "724": 90,
+  "725": 90,
+  "726": 90,
+  "727": 90,
+  "728": 91,
+  "729": 91,
+  "730": 91,
+  "731": 91,
+  "732": 91,
+  "733": 91,
+  "734": 91,
+  "735": 91,
+  "736": 92,
+  "737": 92,
+  "738": 92,
+  "739": 92,
+  "740": 92,
+  "741": 92,
+  "742": 92,
+  "743": 92,
+  "744": 93,
+  "745": 93,
+  "746": 93,
+  "747": 93,
+  "748": 93,
+  "749": 93,
+  "750": 93,
+  "751": 93,
+  "752": 94,
+  "753": 94,
+  "754": 94,
+  "755": 94,
+  "756": 94,
+  "757": 94,
+  "758": 94,
+  "759": 94,
+  "760": 95,
+  "761": 95,
+  "762": 95,
+  "763": 95,
+  "764": 95,
+  "765": 95,
+  "766": 95,
+  "767": 95,
+  "768": 96,
+  "769": 96,
+  "770": 96,
+  "771": 96,
+  "772": 96,
+  "773": 96,
+  "774": 96,
+  "775": 96,
+  "776": 97,
+  "777": 97,
+  "778": 97,
+  "779": 97,
+  "780": 97,
+  "781": 97,
+  "782": 97,
+  "783": 97,
+  "784": 98,
+  "785": 98,
+  "786": 98,
+  "787": 98,
+  "788": 98,
+  "789": 98,
+  "790": 98,
+  "791": 98,
+  "792": 99,
+  "793": 99,
+  "794": 99,
+  "795": 99,
+  "796": 99,
+  "797": 99,
+  "798": 99,
+  "799": 99,
+  "800": 100,
+  "801": 100,
+  "802": 100,
+  "803": 100,
+  "804": 100,
+  "805": 100,
+  "806": 100,
+  "807": 100,
+  "808": 101,
+  "809": 101,
+  "810": 101,
+  "811": 101,
+  "812": 101,
+  "813": 101,
+  "814": 101,
+  "815": 101,
+  "816": 102,
+  "817": 102,
+  "818": 102,
+  "819": 102,
+  "820": 102,
+  "821": 102,
+  "822": 102,
+  "823": 102,
+  "824": 103,
+  "825": 103,
+  "826": 103,
+  "827": 103,
+  "828": 103,
+  "829": 103,
+  "830": 103,
+  "831": 103,
+  "832": 104,
+  "833": 104,
+  "834": 104,
+  "835": 104,
+  "836": 104,
+  "837": 104,
+  "838": 104,
+  "839": 104,
+  "840": 105,
+  "841": 105,
+  "842": 105,
+  "843": 105,
+  "844": 105,
+  "845": 105,
+  "846": 105,
+  "847": 105,
+  "848": 106,
+  "849": 106,
+  "850": 106,
+  "851": 106,
+  "852": 106,
+  "853": 106,
+  "854": 106,
+  "855": 106,
+  "856": 107,
+  "857": 107,
+  "858": 107,
+  "859": 107,
+  "860": 107,
+  "861": 107,
+  "862": 107,
+  "863": 107,
+  "864": 108,
+  "865": 108,
+  "866": 108,
+  "867": 108,
+  "868": 108,
+  "869": 108,
+  "870": 108,
+  "871": 108,
+  "872": 109,
+  "873": 109,
+  "874": 109,
+  "875": 109,
+  "876": 109,
+  "877": 109,
+  "878": 109,
+  "879": 109,
+  "880": 110,
+  "881": 110,
+  "882": 110,
+  "883": 110,
+  "884": 110,
+  "885": 110,
+  "886": 110,
+  "887": 110,
+  "888": 111,
+  "889": 111,
+  "890": 111,
+  "891": 111,
+  "892": 111,
+  "893": 111,
+  "894": 111,
+  "895": 111,
+  "896": 112,
+  "897": 112,
+  "898": 112,
+  "899": 112,
+  "900": 112,
+  "901": 112,
+  "902": 112,
+  "903": 112,
+  "904": 113,
+  "905": 113,
+  "906": 113,
+  "907": 113,
+  "908": 113,
+  "909": 113,
+  "910": 113,
+  "911": 113,
+  "912": 114,
+  "913": 114,
+  "914": 114,
+  "915": 114,
+  "916": 114,
+  "917": 114,
+  "918": 114,
+  "919": 114,
+  "920": 115,
+  "921": 115,
+  "922": 115,
+  "923": 115,
+  "924": 115,
+  "925": 115,
+  "926": 115,
+  "927": 115,
+  "928": 116,
+  "929": 116,
+  "930": 116,
+  "931": 116,
+  "932": 116,
+  "933": 116,
+  "934": 116,
+  "935": 116,
+  "936": 117,
+  "937": 117,
+  "938": 117,
+  "939": 117,
+  "940": 117,
+  "941": 117,
+  "942": 117,
+  "943": 117,
+  "944": 118,
+  "945": 118,
+  "946": 118,
+  "947": 118,
+  "948": 118,
+  "949": 118,
+  "950": 118,
+  "951": 118,
+  "952": 119,
+  "953": 119,
+  "954": 119,
+  "955": 119,
+  "956": 119,
+  "957": 119,
+  "958": 119,
+  "959": 119,
+  "960": 120,
+  "961": 120,
+  "962": 120,
+  "963": 120,
+  "964": 120,
+  "965": 120,
+  "966": 120,
+  "967": 120,
+  "968": 121,
+  "969": 121,
+  "970": 121,
+  "971": 121,
+  "972": 121,
+  "973": 121,
+  "974": 121,
+  "975": 121,
+  "976": 122,
+  "977": 122,
+  "978": 122,
+  "979": 122,
+  "980": 122,
+  "981": 122,
+  "982": 122,
+  "983": 122,
+  "984": 123,
+  "985": 123,
+  "986": 123,
+  "987": 123,
+  "988": 123,
+  "989": 123,
+  "990": 123,
+  "991": 123,
+  "992": 124,
+  "993": 124,
+  "994": 124,
+  "995": 124,
+  "996": 124,
+  "997": 124,
+  "998": 124,
+  "999": 124
 }

backend/src/hand_model_outputsize.py

@@ -1,6 +1,6 @@
+import json
 import torch
 from tqdm import tqdm
-import json
 
 from src.model import handpose_model
 

backend/src/infer_video_d2.py

@@ -9,21 +9,19 @@
 (e.g., .mp4) in a folder.
 """
 
+import argparse
+import copy
 import detectron2
-from detectron2.utils.logger import setup_logger
-from detectron2.config import get_cfg
-from detectron2 import model_zoo
-from detectron2.engine import DefaultPredictor
-
-import subprocess as sp
+import glob
 import numpy as np
-import time
-import argparse
-import sys
 import os
-import glob
-import copy
-
+import subprocess as sp
+import sys
+import time
+from detectron2 import model_zoo
+from detectron2.config import get_cfg
+from detectron2.engine import DefaultPredictor
+from detectron2.utils.logger import setup_logger
 
 def parse_args():
     parser = argparse.ArgumentParser(description='End-to-end inference')
@@ -56,7 +54,6 @@ def parse_args():
     #     sys.exit(1)
     return parser.parse_args()
 
-
 def get_resolution(filename):
     command = ['ffprobe', '-v', 'error', '-select_streams', 'v:0',
                '-show_entries', 'stream=width,height', '-of', 'csv=p=0', filename]
@@ -65,7 +62,6 @@ def get_resolution(filename):
         w, h = line.decode().strip().split(',')
         return int(w), int(h)
 
-
 def read_video(filename):
     w, h = get_resolution(filename)
 
@@ -83,7 +79,6 @@ def read_video(filename):
             break
         yield np.frombuffer(data, dtype='uint8').reshape((h, w, 3))
 
-
 def load_predictor():
     cfg = get_cfg()
     args = parse_args()
@@ -93,7 +88,6 @@ def load_predictor():
     predictor = DefaultPredictor(cfg)
     return predictor
 
-
 def main(args):
     predictor = load_predictor()
     if os.path.isdir(args.im_or_folder):
@@ -151,7 +145,6 @@ def main(args):
 
         np.savez_compressed(out_name, boxes=boxes, segments=segments, keypoints=keypoints, metadata=metadata)
 
-
 def infer_image(im, predictor):
     outputs = predictor(im)['instances'].to('cpu')
 
@@ -207,10 +200,9 @@ def infer_image(im, predictor):
 
     subset = np.arange(20) / 1.0
     subset[-1] = 18
-    subset[-2] = np.sum(kps[3,:])
+    subset[-2] = np.sum(kps[3, :])
     return np.transpose(adjusted_kps[:2]), subset
 
-
 if __name__ == '__main__':
     setup_logger()
     args = parse_args()

backend/src/model.py

@@ -1,9 +1,7 @@
 import torch
-from collections import OrderedDict
-
 import torch
 import torch.nn as nn
-
+from collections import OrderedDict
 
 def make_layers(block, no_relu_layers):
     layers = []
@@ -22,7 +20,6 @@ def make_layers(block, no_relu_layers):
 
     return nn.Sequential(OrderedDict(layers))
 
-
 class bodypose_model(nn.Module):
     def __init__(self):
         super(bodypose_model, self).__init__()
@@ -140,7 +137,6 @@ class bodypose_model(nn.Module):
 
         return out6_1, out6_2
 
-
 class handpose_model(nn.Module):
     def __init__(self):
         super(handpose_model, self).__init__()
@@ -217,7 +213,6 @@ class handpose_model(nn.Module):
         out_stage6 = self.model6(concat_stage6)
         return out_stage6
 
-
 def make_layers_Mconv(block, no_relu_layers):
     modules = []
     for layer_name, v in block.items():
@@ -236,7 +231,6 @@ def make_layers_Mconv(block, no_relu_layers):
         modules.append(nn.Sequential(OrderedDict(layers)))
     return nn.ModuleList(modules)
 
-
 class bodypose_25_model(nn.Module):
     def __init__(self):
         super(bodypose_25_model, self).__init__()

backend/src/util.py

@@ -1,32 +1,31 @@
-import numpy as np
-import math
 import cv2
+import cv2
+import math
 import matplotlib
+import matplotlib.pyplot as plt
+import numpy as np
+import numpy as np
 from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
 from matplotlib.figure import Figure
-import numpy as np
-import matplotlib.pyplot as plt
-import cv2
-
 
 def padRightDownCorner(img, stride, padValue):
     h = img.shape[0]
     w = img.shape[1]
 
     pad = 4 * [None]
-    pad[0] = 0 # up
-    pad[1] = 0 # left
-    pad[2] = 0 if (h % stride == 0) else stride - (h % stride) # down
-    pad[3] = 0 if (w % stride == 0) else stride - (w % stride) # right
+    pad[0] = 0  # up
+    pad[1] = 0  # left
+    pad[2] = 0 if (h % stride == 0) else stride - (h % stride)  # down
+    pad[3] = 0 if (w % stride == 0) else stride - (w % stride)  # right
 
     img_padded = img
-    pad_up = np.tile(img_padded[0:1, :, :]*0 + padValue, (pad[0], 1, 1))
+    pad_up = np.tile(img_padded[0:1, :, :] * 0 + padValue, (pad[0], 1, 1))
     img_padded = np.concatenate((pad_up, img_padded), axis=0)
-    pad_left = np.tile(img_padded[:, 0:1, :]*0 + padValue, (1, pad[1], 1))
+    pad_left = np.tile(img_padded[:, 0:1, :] * 0 + padValue, (1, pad[1], 1))
     img_padded = np.concatenate((pad_left, img_padded), axis=1)
-    pad_down = np.tile(img_padded[-2:-1, :, :]*0 + padValue, (pad[2], 1, 1))
+    pad_down = np.tile(img_padded[-2:-1, :, :] * 0 + padValue, (pad[2], 1, 1))
     img_padded = np.concatenate((img_padded, pad_down), axis=0)
-    pad_right = np.tile(img_padded[:, -2:-1, :]*0 + padValue, (1, pad[3], 1))
+    pad_right = np.tile(img_padded[:, -2:-1, :] * 0 + padValue, (1, pad[3], 1))
     img_padded = np.concatenate((img_padded, pad_right), axis=1)
 
     return img_padded, pad
@@ -90,10 +89,10 @@ def draw_handpose(canvas, all_hand_peaks, show_number=False):
 
     for peaks in all_hand_peaks:
         for ie, e in enumerate(edges):
-            if np.sum(np.all(peaks[e], axis=1)==0)==0:
+            if np.sum(np.all(peaks[e], axis=1) == 0) == 0:
                 x1, y1 = peaks[e[0]]
                 x2, y2 = peaks[e[1]]
-                ax.plot([x1, x2], [y1, y2], color=matplotlib.colors.hsv_to_rgb([ie/float(len(edges)), 1.0, 1.0]))
+                ax.plot([x1, x2], [y1, y2], color=matplotlib.colors.hsv_to_rgb([ie / float(len(edges)), 1.0, 1.0]))
 
         for i, keyponit in enumerate(peaks):
             x, y = keyponit
@@ -111,10 +110,11 @@ def draw_handpose_by_opencv(canvas, peaks, show_number=False):
     # cv2.rectangle(canvas, (x, y), (x+w, y+w), (0, 255, 0), 2, lineType=cv2.LINE_AA)
     # cv2.putText(canvas, 'left' if is_left else 'right', (x, y), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2)
     for ie, e in enumerate(edges):
-        if np.sum(np.all(peaks[e], axis=1)==0)==0:
+        if np.sum(np.all(peaks[e], axis=1) == 0) == 0:
             x1, y1 = peaks[e[0]]
             x2, y2 = peaks[e[1]]
-            cv2.line(canvas, (x1, y1), (x2, y2), matplotlib.colors.hsv_to_rgb([ie/float(len(edges)), 1.0, 1.0])*255, thickness=2)
+            cv2.line(canvas, (x1, y1), (x2, y2), matplotlib.colors.hsv_to_rgb([ie / float(len(edges)), 1.0, 1.0]) * 255,
+                     thickness=2)
 
     for i, keyponit in enumerate(peaks):
         x, y = keyponit
@@ -138,7 +138,7 @@ def handDetect(candidate, subset, oriImg):
         if not (has_left or has_right):
             continue
         hands = []
-        #left hand
+        # left hand
         if has_left:
             left_shoulder_index, left_elbow_index, left_wrist_index = person[[5, 6, 7]]
             x1, y1 = candidate[left_shoulder_index][:2]

backend/src1/__init__.py

@@ -6,6 +6,4 @@ Created on Tue Apr 28 18:47:14 2020
 @author: joe
 """
 
-
-
-#__all__ = ["torch_openpose", "util", "model"]
+# __all__ = ["torch_openpose", "util", "model"]

backend/src1/breakout25.py

@@ -5,20 +5,21 @@ Created on Tue Apr 28 15:13:51 2020
 
 @author: joe
 """
-import numpy as np
 import math
+import numpy as np
 
-jointpairs = [[1,0],[1,2],[2,3],[3,4],[1,5],[5,6],[6,7],[1,8],[8,9],[9,10],[10,11],[8,12],[12,13],[13,14],[0,15],[0,16]\
-              ,[15,17],[16,18],[11,24],[11,22],[14,21],[14,19],[22,23],[19,20]]
-#[[1,0],   [1,2],   [2,3],   [3,4],   [1,5],   [5,6],   [6,7],   [1,8], [8,9], [9,10],[10,11], [8,12],[12,13], [13,14], [0,15],  [0,16]]
+jointpairs = [[1, 0], [1, 2], [2, 3], [3, 4], [1, 5], [5, 6], [6, 7], [1, 8], [8, 9], [9, 10], [10, 11], [8, 12],
+              [12, 13], [13, 14], [0, 15], [0, 16] \
+    , [15, 17], [16, 18], [11, 24], [11, 22], [14, 21], [14, 19], [22, 23], [19, 20]]
+# [[1,0],   [1,2],   [2,3],   [3,4],   [1,5],   [5,6],   [6,7],   [1,8], [8,9], [9,10],[10,11], [8,12],[12,13], [13,14], [0,15],  [0,16]]
 
-#[[30, 31],[14, 15],[16, 17],[18, 19],[22, 23],[24, 25],[26, 27],[0, 1],[6, 7],[2, 3],[4, 5],  [8, 9],[10, 11],[12, 13],[32, 33],[34, 35]]
+# [[30, 31],[14, 15],[16, 17],[18, 19],[22, 23],[24, 25],[26, 27],[0, 1],[6, 7],[2, 3],[4, 5],  [8, 9],[10, 11],[12, 13],[32, 33],[34, 35]]
 
-#[[15,17],[16,18],[11,24],[11,22],[14,21],[14,19],[22,23],[19,20]]
-#[[36,37],[38,39],[50,51],[46,47],[44,45],[40,41],[48,49],[42,43]]
-map25 = [[i,i+1] for i in range(0,52,2)]
+# [[15,17],[16,18],[11,24],[11,22],[14,21],[14,19],[22,23],[19,20]]
+# [[36,37],[38,39],[50,51],[46,47],[44,45],[40,41],[48,49],[42,43]]
+map25 = [[i, i + 1] for i in range(0, 52, 2)]
 
-def findoutmappair(all_peaks,paf):
+def findoutmappair(all_peaks, paf):
     mid_num = 10
     pairmap = []
     for pair in jointpairs:
@@ -30,7 +31,7 @@ def findoutmappair(all_peaks,paf):
         candA = candA[0]
         candB = candB[0]
         startend = list(zip(np.linspace(candA[0], candB[0], num=mid_num), \
-                                            np.linspace(candA[1], candB[1], num=mid_num)))
+                            np.linspace(candA[1], candB[1], num=mid_num)))
 
         vec = np.subtract(candB[:2], candA[:2])
         norm = math.sqrt(vec[0] * vec[0] + vec[1] * vec[1])
@@ -38,20 +39,18 @@ def findoutmappair(all_peaks,paf):
         score = 0.
         tmp = []
         for mp in map25:
-            score_mid = paf[:,:,[mp[0],mp[1]]]
+            score_mid = paf[:, :, [mp[0], mp[1]]]
             vec_x = np.array([score_mid[int(round(startend[I][1])), int(round(startend[I][0])), 0] \
-                                          for I in range(len(startend))])
+                              for I in range(len(startend))])
             vec_y = np.array([score_mid[int(round(startend[I][1])), int(round(startend[I][0])), 1] \
-                                          for I in range(len(startend))])
+                              for I in range(len(startend))])
             score_midpts = np.multiply(vec_x, vec[0]) + np.multiply(vec_y, vec[1])
             score_midpts = score_midpts.sum()
             if score < score_midpts:
                 score = score_midpts
                 tmp = mp
         if score > 0.5:
-            pairmap.append(tmp+[score,])
+            pairmap.append(tmp + [score, ])
         else:
             pairmap.append([])
     return pairmap
-            
-    

backend/src1/model.py

@@ -1,15 +1,14 @@
 import torch
-from collections import OrderedDict
-
 import torch
 import torch.nn as nn
+from collections import OrderedDict
 
-def make_layers(block, no_relu_layers,prelu_layers = []):
+def make_layers(block, no_relu_layers, prelu_layers=[]):
     layers = []
     for layer_name, v in block.items():
         if 'pool' in layer_name:
             layer = nn.MaxPool2d(kernel_size=v[0], stride=v[1],
-                                    padding=v[2])
+                                 padding=v[2])
             layers.append((layer_name, layer))
         else:
             conv2d = nn.Conv2d(in_channels=v[0], out_channels=v[1],
@@ -18,19 +17,19 @@ def make_layers(block, no_relu_layers,prelu_layers = []):
             layers.append((layer_name, conv2d))
             if layer_name not in no_relu_layers:
                 if layer_name not in prelu_layers:
-                    layers.append(('relu_'+layer_name, nn.ReLU(inplace=True)))
+                    layers.append(('relu_' + layer_name, nn.ReLU(inplace=True)))
                 else:
-                    layers.append(('prelu'+layer_name[4:],nn.PReLU(v[1])))
+                    layers.append(('prelu' + layer_name[4:], nn.PReLU(v[1])))
 
     return nn.Sequential(OrderedDict(layers))
 
-def make_layers_Mconv(block,no_relu_layers):
+def make_layers_Mconv(block, no_relu_layers):
     modules = []
     for layer_name, v in block.items():
         layers = []
         if 'pool' in layer_name:
             layer = nn.MaxPool2d(kernel_size=v[0], stride=v[1],
-                                    padding=v[2])
+                                 padding=v[2])
             layers.append((layer_name, layer))
         else:
             conv2d = nn.Conv2d(in_channels=v[0], out_channels=v[1],
@@ -38,197 +37,194 @@ def make_layers_Mconv(block,no_relu_layers):
                                padding=v[4])
             layers.append((layer_name, conv2d))
             if layer_name not in no_relu_layers:
-                layers.append(('Mprelu'+layer_name[5:], nn.PReLU(v[1])))
+                layers.append(('Mprelu' + layer_name[5:], nn.PReLU(v[1])))
         modules.append(nn.Sequential(OrderedDict(layers)))
     return nn.ModuleList(modules)
 
 class bodypose_25_model(nn.Module):
     def __init__(self):
-        super(bodypose_25_model,self).__init__()
+        super(bodypose_25_model, self).__init__()
         # these layers have no relu layer
-        no_relu_layers = ['Mconv7_stage0_L1','Mconv7_stage0_L2',\
-                          'Mconv7_stage1_L1', 'Mconv7_stage1_L2',\
+        no_relu_layers = ['Mconv7_stage0_L1', 'Mconv7_stage0_L2', \
+                          'Mconv7_stage1_L1', 'Mconv7_stage1_L2', \
                           'Mconv7_stage2_L2', 'Mconv7_stage3_L2']
-        prelu_layers = ['conv4_2','conv4_3_CPM','conv4_4_CPM']
+        prelu_layers = ['conv4_2', 'conv4_3_CPM', 'conv4_4_CPM']
         blocks = {}
         block0 = OrderedDict([
-                      ('conv1_1', [3, 64, 3, 1, 1]),
-                      ('conv1_2', [64, 64, 3, 1, 1]),
-                      ('pool1_stage1', [2, 2, 0]),
-                      ('conv2_1', [64, 128, 3, 1, 1]),
-                      ('conv2_2', [128, 128, 3, 1, 1]),
-                      ('pool2_stage1', [2, 2, 0]),
-                      ('conv3_1', [128, 256, 3, 1, 1]),
-                      ('conv3_2', [256, 256, 3, 1, 1]),
-                      ('conv3_3', [256, 256, 3, 1, 1]),
-                      ('conv3_4', [256, 256, 3, 1, 1]),
-                      ('pool3_stage1', [2, 2, 0]),
-                      ('conv4_1', [256, 512, 3, 1, 1]),
-                      ('conv4_2', [512, 512, 3, 1, 1]),
-                      ('conv4_3_CPM', [512, 256, 3, 1, 1]),
-                      ('conv4_4_CPM', [256, 128, 3, 1, 1])
-                  ])
-        self.model0 = make_layers(block0, no_relu_layers,prelu_layers)
-        
-        #L2
-        #stage0
+            ('conv1_1', [3, 64, 3, 1, 1]),
+            ('conv1_2', [64, 64, 3, 1, 1]),
+            ('pool1_stage1', [2, 2, 0]),
+            ('conv2_1', [64, 128, 3, 1, 1]),
+            ('conv2_2', [128, 128, 3, 1, 1]),
+            ('pool2_stage1', [2, 2, 0]),
+            ('conv3_1', [128, 256, 3, 1, 1]),
+            ('conv3_2', [256, 256, 3, 1, 1]),
+            ('conv3_3', [256, 256, 3, 1, 1]),
+            ('conv3_4', [256, 256, 3, 1, 1]),
+            ('pool3_stage1', [2, 2, 0]),
+            ('conv4_1', [256, 512, 3, 1, 1]),
+            ('conv4_2', [512, 512, 3, 1, 1]),
+            ('conv4_3_CPM', [512, 256, 3, 1, 1]),
+            ('conv4_4_CPM', [256, 128, 3, 1, 1])
+        ])
+        self.model0 = make_layers(block0, no_relu_layers, prelu_layers)
+
+        # L2
+        # stage0
         blocks['Mconv1_stage0_L2'] = OrderedDict([
-                    ('Mconv1_stage0_L2_0',[128,96,3,1,1]),
-                    ('Mconv1_stage0_L2_1',[96,96,3,1,1]),
-                    ('Mconv1_stage0_L2_2',[96,96,3,1,1])
-                    ])
-        for i in range(2,6):
+            ('Mconv1_stage0_L2_0', [128, 96, 3, 1, 1]),
+            ('Mconv1_stage0_L2_1', [96, 96, 3, 1, 1]),
+            ('Mconv1_stage0_L2_2', [96, 96, 3, 1, 1])
+        ])
+        for i in range(2, 6):
             blocks['Mconv%d_stage0_L2' % i] = OrderedDict([
-                    ('Mconv%d_stage0_L2_0' % i,[288,96,3,1,1]),
-                    ('Mconv%d_stage0_L2_1' % i,[96,96,3,1,1]),
-                    ('Mconv%d_stage0_L2_2' % i,[96,96,3,1,1])
-              ])
+                ('Mconv%d_stage0_L2_0' % i, [288, 96, 3, 1, 1]),
+                ('Mconv%d_stage0_L2_1' % i, [96, 96, 3, 1, 1]),
+                ('Mconv%d_stage0_L2_2' % i, [96, 96, 3, 1, 1])
+            ])
         blocks['Mconv6_7_stage0_L2'] = OrderedDict([
-                    ('Mconv6_stage0_L2',[288, 256, 1,1,0]),
-                    ('Mconv7_stage0_L2',[256,52,1,1,0])
-              ])
-        #stage1~3
-        for s in range(1,4):
+            ('Mconv6_stage0_L2', [288, 256, 1, 1, 0]),
+            ('Mconv7_stage0_L2', [256, 52, 1, 1, 0])
+        ])
+        # stage1~3
+        for s in range(1, 4):
             blocks['Mconv1_stage%d_L2' % s] = OrderedDict([
-                    ('Mconv1_stage%d_L2_0' % s,[180,128,3,1,1]),
-                    ('Mconv1_stage%d_L2_1' % s,[128,128,3,1,1]),
-                    ('Mconv1_stage%d_L2_2' % s,[128,128,3,1,1])
+                ('Mconv1_stage%d_L2_0' % s, [180, 128, 3, 1, 1]),
+                ('Mconv1_stage%d_L2_1' % s, [128, 128, 3, 1, 1]),
+                ('Mconv1_stage%d_L2_2' % s, [128, 128, 3, 1, 1])
+            ])
+            for i in range(2, 6):
+                blocks['Mconv%d_stage%d_L2' % (i, s)] = OrderedDict([
+                    ('Mconv%d_stage%d_L2_0' % (i, s), [384, 128, 3, 1, 1]),
+                    ('Mconv%d_stage%d_L2_1' % (i, s), [128, 128, 3, 1, 1]),
+                    ('Mconv%d_stage%d_L2_2' % (i, s), [128, 128, 3, 1, 1])
                 ])
-            for i in range(2,6):
-                blocks['Mconv%d_stage%d_L2' % (i,s)] = OrderedDict([
-                        ('Mconv%d_stage%d_L2_0' % (i,s) ,[384,128,3,1,1]),
-                        ('Mconv%d_stage%d_L2_1' % (i,s) ,[128,128,3,1,1]),
-                        ('Mconv%d_stage%d_L2_2' % (i,s) ,[128,128,3,1,1])
-                    ])
             blocks['Mconv6_7_stage%d_L2' % s] = OrderedDict([
-                    ('Mconv6_stage%d_L2' % s,[384,512,1,1,0]),
-                    ('Mconv7_stage%d_L2' % s,[512,52,1,1,0])
-                ])
-        
-        #L1
-        #stage0
-        blocks['Mconv1_stage0_L1'] = OrderedDict([
-                ('Mconv1_stage0_L1_0',[180,96,3,1,1]),
-                ('Mconv1_stage0_L1_1',[96,96,3,1,1]),
-                ('Mconv1_stage0_L1_2',[96,96,3,1,1])
+                ('Mconv6_stage%d_L2' % s, [384, 512, 1, 1, 0]),
+                ('Mconv7_stage%d_L2' % s, [512, 52, 1, 1, 0])
             ])
-        for i in range(2,6):
+
+        # L1
+        # stage0
+        blocks['Mconv1_stage0_L1'] = OrderedDict([
+            ('Mconv1_stage0_L1_0', [180, 96, 3, 1, 1]),
+            ('Mconv1_stage0_L1_1', [96, 96, 3, 1, 1]),
+            ('Mconv1_stage0_L1_2', [96, 96, 3, 1, 1])
+        ])
+        for i in range(2, 6):
             blocks['Mconv%d_stage0_L1' % i] = OrderedDict([
-                    ('Mconv%d_stage0_L1_0' % i,[288,96,3,1,1]),
-                    ('Mconv%d_stage0_L1_1' % i,[96,96,3,1,1]),
-                    ('Mconv%d_stage0_L1_2' % i,[96,96,3,1,1])
-              ])
+                ('Mconv%d_stage0_L1_0' % i, [288, 96, 3, 1, 1]),
+                ('Mconv%d_stage0_L1_1' % i, [96, 96, 3, 1, 1]),
+                ('Mconv%d_stage0_L1_2' % i, [96, 96, 3, 1, 1])
+            ])
         blocks['Mconv6_7_stage0_L1'] = OrderedDict([
-                    ('Mconv6_stage0_L1',[288, 256, 1,1,0]),
-                    ('Mconv7_stage0_L1',[256,26,1,1,0])
-              ])
-        #stage1
+            ('Mconv6_stage0_L1', [288, 256, 1, 1, 0]),
+            ('Mconv7_stage0_L1', [256, 26, 1, 1, 0])
+        ])
+        # stage1
         blocks['Mconv1_stage1_L1'] = OrderedDict([
-                ('Mconv1_stage1_L1_0',[206,128,3,1,1]),
-                ('Mconv1_stage1_L1_1',[128,128,3,1,1]),
-                ('Mconv1_stage1_L1_2',[128,128,3,1,1])
-            ])
-        for i in range(2,6):
+            ('Mconv1_stage1_L1_0', [206, 128, 3, 1, 1]),
+            ('Mconv1_stage1_L1_1', [128, 128, 3, 1, 1]),
+            ('Mconv1_stage1_L1_2', [128, 128, 3, 1, 1])
+        ])
+        for i in range(2, 6):
             blocks['Mconv%d_stage1_L1' % i] = OrderedDict([
-                    ('Mconv%d_stage1_L1_0' % i,[384,128,3,1,1]),
-                    ('Mconv%d_stage1_L1_1' % i,[128,128,3,1,1]),
-                    ('Mconv%d_stage1_L1_2' % i,[128,128,3,1,1])
-                ])
-        blocks['Mconv6_7_stage1_L1'] = OrderedDict([
-                ('Mconv6_stage1_L1',[384,512,1,1,0]),
-                ('Mconv7_stage1_L1',[512,26,1,1,0])
+                ('Mconv%d_stage1_L1_0' % i, [384, 128, 3, 1, 1]),
+                ('Mconv%d_stage1_L1_1' % i, [128, 128, 3, 1, 1]),
+                ('Mconv%d_stage1_L1_2' % i, [128, 128, 3, 1, 1])
             ])
-        
+        blocks['Mconv6_7_stage1_L1'] = OrderedDict([
+            ('Mconv6_stage1_L1', [384, 512, 1, 1, 0]),
+            ('Mconv7_stage1_L1', [512, 26, 1, 1, 0])
+        ])
+
         for k in blocks.keys():
             blocks[k] = make_layers_Mconv(blocks[k], no_relu_layers)
         self.models = nn.ModuleDict(blocks)
-        #self.model_L2_S0_mconv1 = blocks['Mconv1_stage0_L2']
-            
-        
-    def _Mconv_forward(self,x,models):
+        # self.model_L2_S0_mconv1 = blocks['Mconv1_stage0_L2']
+
+    def _Mconv_forward(self, x, models):
         outs = []
         out = x
         for m in models:
             out = m(out)
             outs.append(out)
-        return torch.cat(outs,1)
-        
-    def forward(self,x):
+        return torch.cat(outs, 1)
+
+    def forward(self, x):
         out0 = self.model0(x)
-        #L2
+        # L2
         tout = out0
         for s in range(4):
-            tout = self._Mconv_forward(tout,self.models['Mconv1_stage%d_L2' % s])
-            for v in range(2,6):
-                tout = self._Mconv_forward(tout,self.models['Mconv%d_stage%d_L2' % (v,s)])
+            tout = self._Mconv_forward(tout, self.models['Mconv1_stage%d_L2' % s])
+            for v in range(2, 6):
+                tout = self._Mconv_forward(tout, self.models['Mconv%d_stage%d_L2' % (v, s)])
             tout = self.models['Mconv6_7_stage%d_L2' % s][0](tout)
             tout = self.models['Mconv6_7_stage%d_L2' % s][1](tout)
             outL2 = tout
-            tout = torch.cat([out0,tout],1)
-        #L1 stage0
-        #tout = torch.cat([out0,outL2],1)
+            tout = torch.cat([out0, tout], 1)
+        # L1 stage0
+        # tout = torch.cat([out0,outL2],1)
         tout = self._Mconv_forward(tout, self.models['Mconv1_stage0_L1'])
-        for v in range(2,6):
+        for v in range(2, 6):
             tout = self._Mconv_forward(tout, self.models['Mconv%d_stage0_L1' % v])
         tout = self.models['Mconv6_7_stage0_L1'][0](tout)
         tout = self.models['Mconv6_7_stage0_L1'][1](tout)
         outS0L1 = tout
-        tout = torch.cat([out0,outS0L1,outL2],1)
-        #L1 stage1
+        tout = torch.cat([out0, outS0L1, outL2], 1)
+        # L1 stage1
         tout = self._Mconv_forward(tout, self.models['Mconv1_stage1_L1'])
-        for v in range(2,6):
+        for v in range(2, 6):
             tout = self._Mconv_forward(tout, self.models['Mconv%d_stage1_L1' % v])
         tout = self.models['Mconv6_7_stage1_L1'][0](tout)
         outS1L1 = self.models['Mconv6_7_stage1_L1'][1](tout)
-        
-        return outS1L1,outL2
-        
+
+        return outS1L1, outL2
 
 class bodypose_model(nn.Module):
     def __init__(self):
         super(bodypose_model, self).__init__()
 
         # these layers have no relu layer
-        no_relu_layers = ['conv5_5_CPM_L1', 'conv5_5_CPM_L2', 'Mconv7_stage2_L1',\
-                          'Mconv7_stage2_L2', 'Mconv7_stage3_L1', 'Mconv7_stage3_L2',\
-                          'Mconv7_stage4_L1', 'Mconv7_stage4_L2', 'Mconv7_stage5_L1',\
+        no_relu_layers = ['conv5_5_CPM_L1', 'conv5_5_CPM_L2', 'Mconv7_stage2_L1', \
+                          'Mconv7_stage2_L2', 'Mconv7_stage3_L1', 'Mconv7_stage3_L2', \
+                          'Mconv7_stage4_L1', 'Mconv7_stage4_L2', 'Mconv7_stage5_L1', \
                           'Mconv7_stage5_L2', 'Mconv7_stage6_L1', 'Mconv7_stage6_L1']
         blocks = {}
         block0 = OrderedDict([
-                      ('conv1_1', [3, 64, 3, 1, 1]),
-                      ('conv1_2', [64, 64, 3, 1, 1]),
-                      ('pool1_stage1', [2, 2, 0]),
-                      ('conv2_1', [64, 128, 3, 1, 1]),
-                      ('conv2_2', [128, 128, 3, 1, 1]),
-                      ('pool2_stage1', [2, 2, 0]),
-                      ('conv3_1', [128, 256, 3, 1, 1]),
-                      ('conv3_2', [256, 256, 3, 1, 1]),
-                      ('conv3_3', [256, 256, 3, 1, 1]),
-                      ('conv3_4', [256, 256, 3, 1, 1]),
-                      ('pool3_stage1', [2, 2, 0]),
-                      ('conv4_1', [256, 512, 3, 1, 1]),
-                      ('conv4_2', [512, 512, 3, 1, 1]),
-                      ('conv4_3_CPM', [512, 256, 3, 1, 1]),
-                      ('conv4_4_CPM', [256, 128, 3, 1, 1])
-                  ])
-
+            ('conv1_1', [3, 64, 3, 1, 1]),
+            ('conv1_2', [64, 64, 3, 1, 1]),
+            ('pool1_stage1', [2, 2, 0]),
+            ('conv2_1', [64, 128, 3, 1, 1]),
+            ('conv2_2', [128, 128, 3, 1, 1]),
+            ('pool2_stage1', [2, 2, 0]),
+            ('conv3_1', [128, 256, 3, 1, 1]),
+            ('conv3_2', [256, 256, 3, 1, 1]),
+            ('conv3_3', [256, 256, 3, 1, 1]),
+            ('conv3_4', [256, 256, 3, 1, 1]),
+            ('pool3_stage1', [2, 2, 0]),
+            ('conv4_1', [256, 512, 3, 1, 1]),
+            ('conv4_2', [512, 512, 3, 1, 1]),
+            ('conv4_3_CPM', [512, 256, 3, 1, 1]),
+            ('conv4_4_CPM', [256, 128, 3, 1, 1])
+        ])
 
         # Stage 1
         block1_1 = OrderedDict([
-                        ('conv5_1_CPM_L1', [128, 128, 3, 1, 1]),
-                        ('conv5_2_CPM_L1', [128, 128, 3, 1, 1]),
-                        ('conv5_3_CPM_L1', [128, 128, 3, 1, 1]),
-                        ('conv5_4_CPM_L1', [128, 512, 1, 1, 0]),
-                        ('conv5_5_CPM_L1', [512, 38, 1, 1, 0])
-                    ])
+            ('conv5_1_CPM_L1', [128, 128, 3, 1, 1]),
+            ('conv5_2_CPM_L1', [128, 128, 3, 1, 1]),
+            ('conv5_3_CPM_L1', [128, 128, 3, 1, 1]),
+            ('conv5_4_CPM_L1', [128, 512, 1, 1, 0]),
+            ('conv5_5_CPM_L1', [512, 38, 1, 1, 0])
+        ])
 
         block1_2 = OrderedDict([
-                        ('conv5_1_CPM_L2', [128, 128, 3, 1, 1]),
-                        ('conv5_2_CPM_L2', [128, 128, 3, 1, 1]),
-                        ('conv5_3_CPM_L2', [128, 128, 3, 1, 1]),
-                        ('conv5_4_CPM_L2', [128, 512, 1, 1, 0]),
-                        ('conv5_5_CPM_L2', [512, 19, 1, 1, 0])
-                    ])
+            ('conv5_1_CPM_L2', [128, 128, 3, 1, 1]),
+            ('conv5_2_CPM_L2', [128, 128, 3, 1, 1]),
+            ('conv5_3_CPM_L2', [128, 128, 3, 1, 1]),
+            ('conv5_4_CPM_L2', [128, 512, 1, 1, 0]),
+            ('conv5_5_CPM_L2', [512, 19, 1, 1, 0])
+        ])
         blocks['block1_1'] = block1_1
         blocks['block1_2'] = block1_2
 
@@ -237,24 +233,24 @@ class bodypose_model(nn.Module):
         # Stages 2 - 6
         for i in range(2, 7):
             blocks['block%d_1' % i] = OrderedDict([
-                    ('Mconv1_stage%d_L1' % i, [185, 128, 7, 1, 3]),
-                    ('Mconv2_stage%d_L1' % i, [128, 128, 7, 1, 3]),
-                    ('Mconv3_stage%d_L1' % i, [128, 128, 7, 1, 3]),
-                    ('Mconv4_stage%d_L1' % i, [128, 128, 7, 1, 3]),
-                    ('Mconv5_stage%d_L1' % i, [128, 128, 7, 1, 3]),
-                    ('Mconv6_stage%d_L1' % i, [128, 128, 1, 1, 0]),
-                    ('Mconv7_stage%d_L1' % i, [128, 38, 1, 1, 0])
-                ])
+                ('Mconv1_stage%d_L1' % i, [185, 128, 7, 1, 3]),
+                ('Mconv2_stage%d_L1' % i, [128, 128, 7, 1, 3]),
+                ('Mconv3_stage%d_L1' % i, [128, 128, 7, 1, 3]),
+                ('Mconv4_stage%d_L1' % i, [128, 128, 7, 1, 3]),
+                ('Mconv5_stage%d_L1' % i, [128, 128, 7, 1, 3]),
+                ('Mconv6_stage%d_L1' % i, [128, 128, 1, 1, 0]),
+                ('Mconv7_stage%d_L1' % i, [128, 38, 1, 1, 0])
+            ])
 
             blocks['block%d_2' % i] = OrderedDict([
-                    ('Mconv1_stage%d_L2' % i, [185, 128, 7, 1, 3]),
-                    ('Mconv2_stage%d_L2' % i, [128, 128, 7, 1, 3]),
-                    ('Mconv3_stage%d_L2' % i, [128, 128, 7, 1, 3]),
-                    ('Mconv4_stage%d_L2' % i, [128, 128, 7, 1, 3]),
-                    ('Mconv5_stage%d_L2' % i, [128, 128, 7, 1, 3]),
-                    ('Mconv6_stage%d_L2' % i, [128, 128, 1, 1, 0]),
-                    ('Mconv7_stage%d_L2' % i, [128, 19, 1, 1, 0])
-                ])
+                ('Mconv1_stage%d_L2' % i, [185, 128, 7, 1, 3]),
+                ('Mconv2_stage%d_L2' % i, [128, 128, 7, 1, 3]),
+                ('Mconv3_stage%d_L2' % i, [128, 128, 7, 1, 3]),
+                ('Mconv4_stage%d_L2' % i, [128, 128, 7, 1, 3]),
+                ('Mconv5_stage%d_L2' % i, [128, 128, 7, 1, 3]),
+                ('Mconv6_stage%d_L2' % i, [128, 128, 1, 1, 0]),
+                ('Mconv7_stage%d_L2' % i, [128, 19, 1, 1, 0])
+            ])
 
         for k in blocks.keys():
             blocks[k] = make_layers(blocks[k], no_relu_layers)
@@ -273,7 +269,6 @@ class bodypose_model(nn.Module):
         self.model5_2 = blocks['block5_2']
         self.model6_2 = blocks['block6_2']
 
-
     def forward(self, x):
 
         out1 = self.model0(x)
@@ -301,36 +296,36 @@ class bodypose_model(nn.Module):
         out6_1 = self.model6_1(out6)
         out6_2 = self.model6_2(out6)
 
-        return  out6_2,out6_1
+        return out6_2, out6_1
 
 class handpose_model(nn.Module):
     def __init__(self):
         super(handpose_model, self).__init__()
 
         # these layers have no relu layer
-        no_relu_layers = ['conv6_2_CPM', 'Mconv7_stage2', 'Mconv7_stage3',\
+        no_relu_layers = ['conv6_2_CPM', 'Mconv7_stage2', 'Mconv7_stage3', \
                           'Mconv7_stage4', 'Mconv7_stage5', 'Mconv7_stage6']
         # stage 1
         block1_0 = OrderedDict([
-                ('conv1_1', [3, 64, 3, 1, 1]),
-                ('conv1_2', [64, 64, 3, 1, 1]),
-                ('pool1_stage1', [2, 2, 0]),
-                ('conv2_1', [64, 128, 3, 1, 1]),
-                ('conv2_2', [128, 128, 3, 1, 1]),
-                ('pool2_stage1', [2, 2, 0]),
-                ('conv3_1', [128, 256, 3, 1, 1]),
-                ('conv3_2', [256, 256, 3, 1, 1]),
-                ('conv3_3', [256, 256, 3, 1, 1]),
-                ('conv3_4', [256, 256, 3, 1, 1]),
-                ('pool3_stage1', [2, 2, 0]),
-                ('conv4_1', [256, 512, 3, 1, 1]),
-                ('conv4_2', [512, 512, 3, 1, 1]),
-                ('conv4_3', [512, 512, 3, 1, 1]),
-                ('conv4_4', [512, 512, 3, 1, 1]),
-                ('conv5_1', [512, 512, 3, 1, 1]),
-                ('conv5_2', [512, 512, 3, 1, 1]),
-                ('conv5_3_CPM', [512, 128, 3, 1, 1])
-            ])
+            ('conv1_1', [3, 64, 3, 1, 1]),
+            ('conv1_2', [64, 64, 3, 1, 1]),
+            ('pool1_stage1', [2, 2, 0]),
+            ('conv2_1', [64, 128, 3, 1, 1]),
+            ('conv2_2', [128, 128, 3, 1, 1]),
+            ('pool2_stage1', [2, 2, 0]),
+            ('conv3_1', [128, 256, 3, 1, 1]),
+            ('conv3_2', [256, 256, 3, 1, 1]),
+            ('conv3_3', [256, 256, 3, 1, 1]),
+            ('conv3_4', [256, 256, 3, 1, 1]),
+            ('pool3_stage1', [2, 2, 0]),
+            ('conv4_1', [256, 512, 3, 1, 1]),
+            ('conv4_2', [512, 512, 3, 1, 1]),
+            ('conv4_3', [512, 512, 3, 1, 1]),
+            ('conv4_4', [512, 512, 3, 1, 1]),
+            ('conv5_1', [512, 512, 3, 1, 1]),
+            ('conv5_2', [512, 512, 3, 1, 1]),
+            ('conv5_3_CPM', [512, 128, 3, 1, 1])
+        ])
 
         block1_1 = OrderedDict([
             ('conv6_1_CPM', [128, 512, 1, 1, 0]),
@@ -344,14 +339,14 @@ class handpose_model(nn.Module):
         # stage 2-6
         for i in range(2, 7):
             blocks['block%d' % i] = OrderedDict([
-                    ('Mconv1_stage%d' % i, [150, 128, 7, 1, 3]),
-                    ('Mconv2_stage%d' % i, [128, 128, 7, 1, 3]),
-                    ('Mconv3_stage%d' % i, [128, 128, 7, 1, 3]),
-                    ('Mconv4_stage%d' % i, [128, 128, 7, 1, 3]),
-                    ('Mconv5_stage%d' % i, [128, 128, 7, 1, 3]),
-                    ('Mconv6_stage%d' % i, [128, 128, 1, 1, 0]),
-                    ('Mconv7_stage%d' % i, [128, 22, 1, 1, 0])
-                ])
+                ('Mconv1_stage%d' % i, [150, 128, 7, 1, 3]),
+                ('Mconv2_stage%d' % i, [128, 128, 7, 1, 3]),
+                ('Mconv3_stage%d' % i, [128, 128, 7, 1, 3]),
+                ('Mconv4_stage%d' % i, [128, 128, 7, 1, 3]),
+                ('Mconv5_stage%d' % i, [128, 128, 7, 1, 3]),
+                ('Mconv6_stage%d' % i, [128, 128, 1, 1, 0]),
+                ('Mconv7_stage%d' % i, [128, 22, 1, 1, 0])
+            ])
 
         for k in blocks.keys():
             blocks[k] = make_layers(blocks[k], no_relu_layers)
@@ -378,5 +373,3 @@ class handpose_model(nn.Module):
         concat_stage6 = torch.cat([out_stage5, out1_0], 1)
         out_stage6 = self.model6(concat_stage6)
         return out_stage6
-
-

backend/src1/torch_openpose.py

@@ -1,11 +1,11 @@
 import cv2
-import numpy as np
 import math
-from scipy.ndimage.filters import gaussian_filter
+import numpy as np
 import torch
+from scipy.ndimage.filters import gaussian_filter
 
 from src1 import util
-from src1.model import bodypose_model,bodypose_25_model
+from src1.model import bodypose_model, bodypose_25_model
 
 model_coco = 'model/body_coco.pth'
 model_body25 = 'model/body_25.pth'
@@ -26,21 +26,20 @@ class torch_openpose(object):
             self.model = self.model.cuda()
         self.model.eval()
 
-        if self.njoint == 19:  #coco
+        if self.njoint == 19:  # coco
             self.limbSeq = [[1, 2], [1, 5], [2, 3], [3, 4], [5, 6], [6, 7], [1, 8], [8, 9], \
-                   [9, 10], [1, 11], [11, 12], [12, 13], [1, 0], [0, 14], [14, 16], \
-                   [0, 15], [15, 17]]
-            self.mapIdx = [[12, 13],[20, 21],[14, 15],[16, 17],[22, 23],[24, 25],[0, 1],[2, 3],\
-                           [4, 5],[6, 7],[8, 9],[10, 11],[28, 29],[30, 31],[34, 35],[32, 33],\
-                               [36, 37]]
-        elif self.njoint == 26:  #body_25
-            self.limbSeq = [[1,0],[1,2],[2,3],[3,4],[1,5],[5,6],[6,7],[1,8],[8,9],[9,10],\
-                            [10,11],[8,12],[12,13],[13,14],[0,15],[0,16],[15,17],[16,18],\
-                                [11,24],[11,22],[14,21],[14,19],[22,23],[19,20]]
-            self.mapIdx = [[30, 31],[14, 15],[16, 17],[18, 19],[22, 23],[24, 25],[26, 27],[0, 1],[6, 7],\
-                           [2, 3],[4, 5],  [8, 9],[10, 11],[12, 13],[32, 33],[34, 35],[36,37],[38,39],\
-                               [50,51],[46,47],[44,45],[40,41],[48,49],[42,43]]
-
+                            [9, 10], [1, 11], [11, 12], [12, 13], [1, 0], [0, 14], [14, 16], \
+                            [0, 15], [15, 17]]
+            self.mapIdx = [[12, 13], [20, 21], [14, 15], [16, 17], [22, 23], [24, 25], [0, 1], [2, 3], \
+                           [4, 5], [6, 7], [8, 9], [10, 11], [28, 29], [30, 31], [34, 35], [32, 33], \
+                           [36, 37]]
+        elif self.njoint == 26:  # body_25
+            self.limbSeq = [[1, 0], [1, 2], [2, 3], [3, 4], [1, 5], [5, 6], [6, 7], [1, 8], [8, 9], [9, 10], \
+                            [10, 11], [8, 12], [12, 13], [13, 14], [0, 15], [0, 16], [15, 17], [16, 18], \
+                            [11, 24], [11, 22], [14, 21], [14, 19], [22, 23], [19, 20]]
+            self.mapIdx = [[30, 31], [14, 15], [16, 17], [18, 19], [22, 23], [24, 25], [26, 27], [0, 1], [6, 7], \
+                           [2, 3], [4, 5], [8, 9], [10, 11], [12, 13], [32, 33], [34, 35], [36, 37], [38, 39], \
+                           [50, 51], [46, 47], [44, 45], [40, 41], [48, 49], [42, 43]]
 
     def __call__(self, oriImg):
         # scale_search = [0.5, 1.0, 1.5, 2.0]
@@ -104,7 +103,8 @@ class torch_openpose(object):
             map_down[:, :-1] = one_heatmap[:, 1:]
 
             peaks_binary = np.logical_and.reduce(
-                (one_heatmap >= map_left, one_heatmap >= map_right, one_heatmap >= map_up, one_heatmap >= map_down, one_heatmap > thre1))
+                (one_heatmap >= map_left, one_heatmap >= map_right, one_heatmap >= map_up, one_heatmap >= map_down,
+                 one_heatmap > thre1))
             peaks = list(zip(np.nonzero(peaks_binary)[1], np.nonzero(peaks_binary)[0]))  # note reverse
             peaks_with_score = [x + (map_ori[x[1], x[0]],) for x in peaks]
             peak_id = range(peak_counter, peak_counter + len(peaks))
@@ -228,10 +228,8 @@ class torch_openpose(object):
                 if po >= 0:
                     joint = list(candidate[int(po)][:3])
                 else:
-                    joint = [0.,0.,0.]
+                    joint = [0., 0., 0.]
                 pose.append(joint)
             poses.append(pose)
 
         return poses
-
-

backend/src1/util.py

@@ -1,26 +1,25 @@
-import numpy as np
-import math
 import cv2
-
+import math
+import numpy as np
 
 # draw the body keypoint and lims
-def draw_bodypose(img, poses,model_type = 'coco'):
+def draw_bodypose(img, poses, model_type='coco'):
     stickwidth = 4
-    
+
     limbSeq = [[1, 2], [1, 5], [2, 3], [3, 4], [5, 6], [6, 7], [1, 8], [8, 9], \
-                   [9, 10], [1, 11], [11, 12], [12, 13], [1, 0], [0, 14], [14, 16], \
-                   [0, 15], [15, 17]]
+               [9, 10], [1, 11], [11, 12], [12, 13], [1, 0], [0, 14], [14, 16], \
+               [0, 15], [15, 17]]
     njoint = 18
-    if model_type == 'body_25':    
-        limbSeq = [[1,0],[1,2],[2,3],[3,4],[1,5],[5,6],[6,7],[1,8],[8,9],[9,10],\
-                            [10,11],[8,12],[12,13],[13,14],[0,15],[0,16],[15,17],[16,18],\
-                                [11,24],[11,22],[14,21],[14,19],[22,23],[19,20]]
+    if model_type == 'body_25':
+        limbSeq = [[1, 0], [1, 2], [2, 3], [3, 4], [1, 5], [5, 6], [6, 7], [1, 8], [8, 9], [9, 10], \
+                   [10, 11], [8, 12], [12, 13], [13, 14], [0, 15], [0, 16], [15, 17], [16, 18], \
+                   [11, 24], [11, 22], [14, 21], [14, 19], [22, 23], [19, 20]]
         njoint = 25
 
     colors = [[255, 0, 0], [255, 85, 0], [255, 170, 0], [255, 255, 0], [170, 255, 0], [85, 255, 0], [0, 255, 0], \
               [0, 255, 85], [0, 255, 170], [0, 255, 255], [0, 170, 255], [0, 85, 255], [0, 0, 255], [85, 0, 255], \
-              [170, 0, 255], [255, 0, 255], [255, 0, 170], [255, 0, 85], [255,255,0], [255,255,85], [255,255,170],\
-                  [255,255,255],[170,255,255],[85,255,255],[0,255,255]]
+              [170, 0, 255], [255, 0, 255], [255, 0, 170], [255, 0, 85], [255, 255, 0], [255, 255, 85], [255, 255, 170], \
+              [255, 255, 255], [170, 255, 255], [85, 255, 255], [0, 255, 255]]
     for i in range(njoint):
         for n in range(len(poses)):
             pose = poses[n][i]
@@ -28,16 +27,16 @@ def draw_bodypose(img, poses,model_type = 'coco'):
                 continue
             x, y = pose[:2]
             cv2.circle(img, (int(x), int(y)), 4, colors[i], thickness=-1)
-    
+
     for pose in poses:
-        for limb,color in zip(limbSeq,colors):
+        for limb, color in zip(limbSeq, colors):
             p1 = pose[limb[0]]
             p2 = pose[limb[1]]
-            if p1[2] <=0 or p2[2] <= 0:
+            if p1[2] <= 0 or p2[2] <= 0:
                 continue
             cur_canvas = img.copy()
-            X = [p1[1],p2[1]]
-            Y = [p1[0],p2[0]]
+            X = [p1[1], p2[1]]
+            Y = [p1[0], p2[0]]
             mX = np.mean(X)
             mY = np.mean(Y)
             length = ((X[0] - X[1]) ** 2 + (Y[0] - Y[1]) ** 2) ** 0.5
@@ -45,7 +44,7 @@ def draw_bodypose(img, poses,model_type = 'coco'):
             polygon = cv2.ellipse2Poly((int(mY), int(mX)), (int(length / 2), stickwidth), int(angle), 0, 360, 1)
             cv2.fillConvexPoly(cur_canvas, polygon, color)
             img = cv2.addWeighted(img, 0.4, cur_canvas, 0.6, 0)
-   
+
     return img
 
 def padRightDownCorner(img, stride, padValue):
@@ -53,19 +52,19 @@ def padRightDownCorner(img, stride, padValue):
     w = img.shape[1]
 
     pad = 4 * [None]
-    pad[0] = 0 # up
-    pad[1] = 0 # left
-    pad[2] = 0 if (h % stride == 0) else stride - (h % stride) # down
-    pad[3] = 0 if (w % stride == 0) else stride - (w % stride) # right
+    pad[0] = 0  # up
+    pad[1] = 0  # left
+    pad[2] = 0 if (h % stride == 0) else stride - (h % stride)  # down
+    pad[3] = 0 if (w % stride == 0) else stride - (w % stride)  # right
 
     img_padded = img
-    pad_up = np.tile(img_padded[0:1, :, :]*0 + padValue, (pad[0], 1, 1))
+    pad_up = np.tile(img_padded[0:1, :, :] * 0 + padValue, (pad[0], 1, 1))
     img_padded = np.concatenate((pad_up, img_padded), axis=0)
-    pad_left = np.tile(img_padded[:, 0:1, :]*0 + padValue, (1, pad[1], 1))
+    pad_left = np.tile(img_padded[:, 0:1, :] * 0 + padValue, (1, pad[1], 1))
     img_padded = np.concatenate((pad_left, img_padded), axis=1)
-    pad_down = np.tile(img_padded[-2:-1, :, :]*0 + padValue, (pad[2], 1, 1))
+    pad_down = np.tile(img_padded[-2:-1, :, :] * 0 + padValue, (pad[2], 1, 1))
     img_padded = np.concatenate((img_padded, pad_down), axis=0)
-    pad_right = np.tile(img_padded[:, -2:-1, :]*0 + padValue, (1, pad[3], 1))
+    pad_right = np.tile(img_padded[:, -2:-1, :] * 0 + padValue, (1, pad[3], 1))
     img_padded = np.concatenate((img_padded, pad_right), axis=1)
 
     return img_padded, pad

backend/video_pose/CODE_OF_CONDUCT.md

@@ -1,2 +1,5 @@
 # Code of Conduct
-Facebook has adopted a Code of Conduct that we expect project participants to adhere to. Please [read the full text](https://code.facebook.com/codeofconduct) so that you can understand what actions will and will not be tolerated.
+
+Facebook has adopted a Code of Conduct that we expect project participants to adhere to.
+Please [read the full text](https://code.facebook.com/codeofconduct) so that you can understand what actions will and
+will not be tolerated.

backend/video_pose/CONTRIBUTING.md

@@ -1,8 +1,10 @@
 # Contributing
+
 We want to make contributing to this project as easy and transparent as
 possible.
 
 ## Pull Requests
+
 We actively welcome your pull requests.
 
 1. Fork the repo and create your branch from `master`.
@@ -13,18 +15,22 @@ We actively welcome your pull requests.
 6. If you haven't already, complete the Contributor License Agreement ("CLA").
 
 ## Contributor License Agreement ("CLA")
+
 In order to accept your pull request, we need you to submit a CLA. You only need
 to do this once to work on any of Facebook's open source projects.
 
 Complete your CLA here: <https://code.facebook.com/cla>
 
 ## Issues
+
 We use GitHub issues to track public bugs. Please ensure your description is
 clear and has sufficient instructions to be able to reproduce the issue.
 
 ## Coding Style
+
 We follow the [PEP 8](https://www.python.org/dev/peps/pep-0008/) style guidelines.
 
 ## License
+
 By contributing to this project, you agree that your contributions will be licensed
 under the LICENSE file in the root directory of this source tree.

backend/video_pose/DATASETS.md

@@ -1,17 +1,27 @@
 # Dataset setup
 
 ## Human3.6M
-We provide two ways to set up the Human3.6M dataset on our pipeline. You can either convert the original dataset (recommended) or use the [dataset preprocessed by Martinez et al.](https://github.com/una-dinosauria/3d-pose-baseline) (no longer available as of May 22nd, 2020). The two methods produce the same result. After this step, you should end up with two files in the `data` directory: `data_3d_h36m.npz` for the 3D poses, and `data_2d_h36m_gt.npz` for the ground-truth 2D poses.
+
+We provide two ways to set up the Human3.6M dataset on our pipeline. You can either convert the original dataset (
+recommended) or use the [dataset preprocessed by Martinez et al.](https://github.com/una-dinosauria/3d-pose-baseline) (
+no longer available as of May 22nd, 2020). The two methods produce the same result. After this step, you should end up
+with two files in the `data` directory: `data_3d_h36m.npz` for the 3D poses, and `data_2d_h36m_gt.npz` for the
+ground-truth 2D poses.
 
 ### Setup from original source (recommended)
+
 **Update:** we have updated the instructions to simplify the procedure. MATLAB is no longer required for this step.
 
-Register to the [Human3.6m website](http://vision.imar.ro/human3.6m/) website (or login if you already have an account) and download the dataset in its original format. You only need to download *Poses -> D3 Positions* for each subject (1, 5, 6, 7, 8, 9, 11)
+Register to the [Human3.6m website](http://vision.imar.ro/human3.6m/) website (or login if you already have an account)
+and download the dataset in its original format. You only need to download *Poses -> D3 Positions* for each subject (1,
+5, 6, 7, 8, 9, 11)
 
 ##### Instructions without MATLAB (recommended)
+
 You first need to install `cdflib` Python library via `pip install cdflib`.
 
-Extract the archives named `Poses_D3_Positions_S*.tgz` (subjects 1, 5, 6, 7, 8, 9, 11) to a common directory. Your directory tree should look like this:
+Extract the archives named `Poses_D3_Positions_S*.tgz` (subjects 1, 5, 6, 7, 8, 9, 11) to a common directory. Your
+directory tree should look like this:
 
 ```
 /path/to/dataset/S1/MyPoseFeatures/D3_Positions/Directions 1.cdf
@@ -20,6 +30,7 @@ Extract the archives named `Poses_D3_Positions_S*.tgz` (subjects 1, 5, 6, 7, 8,
 ```
 
 Then, run the preprocessing script:
+
 ```sh
 cd data
 python prepare_data_h36m.py --from-source-cdf /path/to/dataset
@@ -29,7 +40,11 @@ cd ..
 If everything goes well, you are ready to go.
 
 ##### Instructions with MATLAB (old instructions)
-First, we need to convert the 3D poses from `.cdf` to `.mat`, so they can be loaded from Python scripts. To this end, we have provided the MATLAB script `convert_cdf_to_mat.m` in the `data` directory. Extract the archives named `Poses_D3_Positions_S*.tgz` (subjects 1, 5, 6, 7, 8, 9, 11) to a directory named `pose`, and set up your directory tree so that it looks like this:
+
+First, we need to convert the 3D poses from `.cdf` to `.mat`, so they can be loaded from Python scripts. To this end, we
+have provided the MATLAB script `convert_cdf_to_mat.m` in the `data` directory. Extract the archives
+named `Poses_D3_Positions_S*.tgz` (subjects 1, 5, 6, 7, 8, 9, 11) to a directory named `pose`, and set up your directory
+tree so that it looks like this:
 
 ```
 /path/to/dataset/convert_cdf_to_mat.m
@@ -37,9 +52,11 @@ First, we need to convert the 3D poses from `.cdf` to `.mat`, so they can be loa
 /path/to/dataset/pose/S1/MyPoseFeatures/D3_Positions/Directions.cdf
 ...
 ```
+
 Then run `convert_cdf_to_mat.m` from MATLAB.
 
 Finally, run the Python conversion script specifying the dataset path:
+
 ```sh
 cd data
 python prepare_data_h36m.py --from-source /path/to/dataset/pose
@@ -47,9 +64,14 @@ cd ..
 ```
 
 ### Setup from preprocessed dataset (old instructions)
-**Update:** the link to the preprocessed dataset is no longer available; please use the procedure above. These instructions have been kept for backwards compatibility in case you already have a copy of this archive. All procedures produce the same result.
 
-Download the [~~h36m.zip archive~~](https://www.dropbox.com/s/e35qv3n6zlkouki/h36m.zip) (source: [3D pose baseline repository](https://github.com/una-dinosauria/3d-pose-baseline)) to the `data` directory, and run the conversion script from the same directory. This step does not require any additional dependency.
+**Update:** the link to the preprocessed dataset is no longer available; please use the procedure above. These
+instructions have been kept for backwards compatibility in case you already have a copy of this archive. All procedures
+produce the same result.
+
+Download the [~~h36m.zip archive~~](https://www.dropbox.com/s/e35qv3n6zlkouki/h36m.zip) (
+source: [3D pose baseline repository](https://github.com/una-dinosauria/3d-pose-baseline)) to the `data` directory, and
+run the conversion script from the same directory. This step does not require any additional dependency.
 
 ```sh
 cd data
@@ -59,6 +81,7 @@ cd ..
 ```
 
 ## 2D detections for Human3.6M
+
 We provide support for the following 2D detections:
 
 - `gt`: ground-truth 2D poses, extracted through the camera projection parameters.
@@ -66,14 +89,21 @@ We provide support for the following 2D detections:
 - `sh_ft_h36m`: Stacked Hourglass detections, fine-tuned on Human3.6M.
 - `detectron_pt_h36m`: Detectron (Mask R-CNN) detections (model pretrained on COCO, no fine tuning).
 - `detectron_ft_h36m`: Detectron (Mask R-CNN) detections, fine-tuned on Human3.6M.
-- `cpn_ft_h36m_dbb`: Cascaded Pyramid Network detections, fine-tuned on Human3.6M. Bounding boxes from `detectron_ft_h36m`.
+- `cpn_ft_h36m_dbb`: Cascaded Pyramid Network detections, fine-tuned on Human3.6M. Bounding boxes
+  from `detectron_ft_h36m`.
 - User-supplied (see below).
 
-The 2D detection source is specified through the `--keypoints` parameter, which loads the file `data_2d_DATASET_DETECTION.npz` from the `data` directory, where `DATASET` is the dataset name (e.g. `h36m`) and `DETECTION` is the 2D detection source (e.g. `sh_pt_mpii`). Since all the files are encoded according to the same format, it is trivial to create a custom set of 2D detections.
+The 2D detection source is specified through the `--keypoints` parameter, which loads the
+file `data_2d_DATASET_DETECTION.npz` from the `data` directory, where `DATASET` is the dataset name (e.g. `h36m`)
+and `DETECTION` is the 2D detection source (e.g. `sh_pt_mpii`). Since all the files are encoded according to the same
+format, it is trivial to create a custom set of 2D detections.
 
-Ground-truth poses (`gt`) have already been extracted by the previous step. The other detections must be downloaded manually (see instructions below). You only need to download the detections you want to use. For reference, our best results on Human3.6M are achieved by `cpn_ft_h36m_dbb`.
+Ground-truth poses (`gt`) have already been extracted by the previous step. The other detections must be downloaded
+manually (see instructions below). You only need to download the detections you want to use. For reference, our best
+results on Human3.6M are achieved by `cpn_ft_h36m_dbb`.
 
 ### Mask R-CNN and CPN detections
+
 You can download these directly and put them in the `data` directory. We recommend starting with:
 
 ```sh
@@ -83,9 +113,12 @@ wget https://dl.fbaipublicfiles.com/video-pose-3d/data_2d_h36m_detectron_ft_h36m
 cd ..
 ```
 
-These detections have been produced by models fine-tuned on Human3.6M. We adopted the usual protocol of fine-tuning on 5 subjects (S1, S5, S6, S7, and S8). We also included detections from the unlabeled subjects S2, S3, S4, which can be loaded by our framework for semi-supervised experimentation.
+These detections have been produced by models fine-tuned on Human3.6M. We adopted the usual protocol of fine-tuning on 5
+subjects (S1, S5, S6, S7, and S8). We also included detections from the unlabeled subjects S2, S3, S4, which can be
+loaded by our framework for semi-supervised experimentation.
 
 Optionally, you can download the Mask R-CNN detections without fine-tuning if you want to experiment with these:
+
 ```sh
 cd data
 wget https://dl.fbaipublicfiles.com/video-pose-3d/data_2d_h36m_detectron_pt_coco.npz
@@ -93,7 +126,13 @@ cd ..
 ```
 
 ### Stacked Hourglass detections
-These detections (both pretrained and fine-tuned) are provided by [Martinez et al.](https://github.com/una-dinosauria/3d-pose-baseline) in their repository on 3D human pose estimation. The 2D poses produced by the pretrained model are in the same archive as the dataset ([h36m.zip](https://www.dropbox.com/s/e35qv3n6zlkouki/h36m.zip)). The fine-tuned poses can be downloaded [here](https://drive.google.com/open?id=0BxWzojlLp259S2FuUXJ6aUNxZkE). Put the two archives in the `data` directory and run:
+
+These detections (both pretrained and fine-tuned) are provided
+by [Martinez et al.](https://github.com/una-dinosauria/3d-pose-baseline) in their repository on 3D human pose
+estimation. The 2D poses produced by the pretrained model are in the same archive as the
+dataset ([h36m.zip](https://www.dropbox.com/s/e35qv3n6zlkouki/h36m.zip)). The fine-tuned poses can be
+downloaded [here](https://drive.google.com/open?id=0BxWzojlLp259S2FuUXJ6aUNxZkE). Put the two archives in the `data`
+directory and run:
 
 ```sh
 cd data
@@ -103,31 +142,56 @@ cd ..
 ```
 
 ## HumanEva-I
-For HumanEva, you need the original dataset and MATLAB. We provide a MATLAB script to extract the revelant parts of the dataset automatically.
 
-1. Download the [HumanEva-I dataset](http://humaneva.is.tue.mpg.de/datasets_human_1) and extract it.
-2. Download the official [source code v1.1 beta](http://humaneva.is.tue.mpg.de/main/download?file=Release_Code_v1_1_beta.zip) and extract it where you extracted the dataset.
-3. Copy the contents of the directory `Release_Code_v1_1_beta\HumanEva_I` to the root of the source tree (`Release_Code_v1_1_beta/`).
-4. Download the [critical dataset update](http://humaneva.is.tue.mpg.de/main/download?file=Critical_Update_OFS_files.zip) and apply it.
-5. **Important:** for visualization purposes, the original code requires an old library named *dxAvi*, which is used for decoding XVID videos. A precompiled binary for 32-bit architectures is already included, but if you are running MATLAB on a 64-bit system, the code will not work. You can either recompile *dxAvi* library for x64, or bypass it entirely, since we are not using visualization features in our conversion script. To this end, you can patch `@sync_stream/sync_stream.m`, replacing line 202: `ImageStream(I) = image_stream(image_paths{I}, start_image_offset(I));` with `ImageStream(I) = 0;`
-6. Now you can copy our script `ConvertHumanEva.m` (from `data/`) to `Release_Code_v1_1_beta/`, and run it. It will create a directory named `converted_15j`, which contains the converted 2D/3D ground-truth poses on a 15-joint skeleton.
-7. **Optional:** if you want to experiment with a 20-joint skeleton, change `N_JOINTS` to 20 in `ConvertHumanEva.m`, and repeat the process. It will create a directory named `converted_20j`. Adapt next steps accordingly.
+For HumanEva, you need the original dataset and MATLAB. We provide a MATLAB script to extract the revelant parts of the
+dataset automatically.
 
-If you get warnings about mocap errors or dropped frames, this is normal. The HumanEva dataset contains some invalid frames due to occlusions, which are simply discarded. Since we work with videos (and not individual frames), we try to minimize the impact of this issue by grouping valid sequences into contiguous chunks.
+1. Download the [HumanEva-I dataset](http://humaneva.is.tue.mpg.de/datasets_human_1) and extract it.
+2. Download the
+   official [source code v1.1 beta](http://humaneva.is.tue.mpg.de/main/download?file=Release_Code_v1_1_beta.zip) and
+   extract it where you extracted the dataset.
+3. Copy the contents of the directory `Release_Code_v1_1_beta\HumanEva_I` to the root of the source
+   tree (`Release_Code_v1_1_beta/`).
+4. Download
+   the [critical dataset update](http://humaneva.is.tue.mpg.de/main/download?file=Critical_Update_OFS_files.zip) and
+   apply it.
+5. **Important:** for visualization purposes, the original code requires an old library named *dxAvi*, which is used for
+   decoding XVID videos. A precompiled binary for 32-bit architectures is already included, but if you are running
+   MATLAB on a 64-bit system, the code will not work. You can either recompile *dxAvi* library for x64, or bypass it
+   entirely, since we are not using visualization features in our conversion script. To this end, you can
+   patch `@sync_stream/sync_stream.m`, replacing line
+   202: `ImageStream(I) = image_stream(image_paths{I}, start_image_offset(I));` with `ImageStream(I) = 0;`
+6. Now you can copy our script `ConvertHumanEva.m` (from `data/`) to `Release_Code_v1_1_beta/`, and run it. It will
+   create a directory named `converted_15j`, which contains the converted 2D/3D ground-truth poses on a 15-joint
+   skeleton.
+7. **Optional:** if you want to experiment with a 20-joint skeleton, change `N_JOINTS` to 20 in `ConvertHumanEva.m`, and
+   repeat the process. It will create a directory named `converted_20j`. Adapt next steps accordingly.
+
+If you get warnings about mocap errors or dropped frames, this is normal. The HumanEva dataset contains some invalid
+frames due to occlusions, which are simply discarded. Since we work with videos (and not individual frames), we try to
+minimize the impact of this issue by grouping valid sequences into contiguous chunks.
 
 Finally, run the Python script to produce the final files:
+
 ```
 python prepare_data_humaneva.py -p /path/to/dataset/Release_Code_v1_1_beta/converted_15j --convert-3d
 ```
-You should end up with two files in the `data` directory: `data_3d_humaneva15.npz` for the 3D poses, and `data_2d_humaneva15_gt.npz` for the ground-truth 2D poses.
+
+You should end up with two files in the `data` directory: `data_3d_humaneva15.npz` for the 3D poses,
+and `data_2d_humaneva15_gt.npz` for the ground-truth 2D poses.
 
 ### 2D detections for HumanEva-I
+
 We provide support for the following 2D detections:
 
 - `gt`: ground-truth 2D poses, extracted through camera projection.
 - `detectron_pt_coco`: Detectron (Mask R-CNN) detections, pretrained on COCO.
 
-Since HumanEva is very small, we do not fine-tune the pretrained models. As before, you can download Mask R-CNN detections from AWS (`data_2d_humaneva15_detectron_pt_coco.npz`, which must be copied to `data/`). As before, we have included detections for unlabeled subjects/actions. These begin with the prefix `Unlabeled/`. Chunks that correspond to corrupted motion capture streams are also marked as unlabeled.
+Since HumanEva is very small, we do not fine-tune the pretrained models. As before, you can download Mask R-CNN
+detections from AWS (`data_2d_humaneva15_detectron_pt_coco.npz`, which must be copied to `data/`). As before, we have
+included detections for unlabeled subjects/actions. These begin with the prefix `Unlabeled/`. Chunks that correspond to
+corrupted motion capture streams are also marked as unlabeled.
+
 ```sh
 cd data
 wget https://dl.fbaipublicfiles.com/video-pose-3d/data_2d_humaneva15_detectron_pt_coco.npz

backend/video_pose/DOCUMENTATION.md

@@ -1,118 +1,197 @@
 # Documentation
-This guide explains in depth all the features of this framework. Make sure you have read the quick start guide in [`README.md`](README.md) before proceeding.
+
+This guide explains in depth all the features of this framework. Make sure you have read the quick start guide
+in [`README.md`](README.md) before proceeding.
 
 ## Training
-By default, the script `run.py` runs in training mode. The list of command-line arguments is defined in `common/arguments.py`.
+
+By default, the script `run.py` runs in training mode. The list of command-line arguments is defined
+in `common/arguments.py`.
 
 - `-h`: shows the help / list of parameters.
-- `-d` or `--dataset`: specifies the dataset to use (`h36m` or `humaneva15`). Default: `h36m`. If you converted the 20-joint HumanEva skeleton, you can also use `humaneva20`.
+- `-d` or `--dataset`: specifies the dataset to use (`h36m` or `humaneva15`). Default: `h36m`. If you converted the
+  20-joint HumanEva skeleton, you can also use `humaneva20`.
 - `-k` or `--keypoints`: specifies the 2D detections to use. Default: `cpn_ft_h36m_dbb` (CPN fine-tuned on Human 3.6M).
 - `-c` or `--checkpoint`: specifies the directory where checkpoints are saved/read. Default: `checkpoint`.
 - `--checkpoint-frequency`: save checkpoints every N epochs. Default: `10`.
-- `-r` or `--resume`: resume training from a particular checkpoint (you should only specify the file name, not the path), e.g. `epoch_10.bin`.
-- `-str` or `--subjects-train`: specifies the list of subjects on which the model is trained, separated by commas. Default: `S1,S5,S6,S7,S8`. For HumanEva, you may want to specify these manually.
-- `-ste` or `--subjects-test`: specifies the list of subjects on which the model is tested at the end of each epoch (and in the final evaluation), separated by comma. Default: `S9,S11`. For HumanEva, you may want to specify these manually.
-- `-a` or `--actions`: select only a subset of actions, separated by commas. E.g. `Walk,Jog`. By default, all actions are used.
+- `-r` or `--resume`: resume training from a particular checkpoint (you should only specify the file name, not the path)
+  , e.g. `epoch_10.bin`.
+- `-str` or `--subjects-train`: specifies the list of subjects on which the model is trained, separated by commas.
+  Default: `S1,S5,S6,S7,S8`. For HumanEva, you may want to specify these manually.
+- `-ste` or `--subjects-test`: specifies the list of subjects on which the model is tested at the end of each epoch (and
+  in the final evaluation), separated by comma. Default: `S9,S11`. For HumanEva, you may want to specify these manually.
+- `-a` or `--actions`: select only a subset of actions, separated by commas. E.g. `Walk,Jog`. By default, all actions
+  are used.
 - `-e` or `--epochs`: train for N epochs, i.e. N passes over the entire training set. Default: `60`.
 - `--no-eval`: disable testing at the end of each epoch (marginal speed up). By default, testing is enabled.
-- `--export-training-curves`: export training curves as PNG images after every epoch. They are saved in the checkpoint directory. Default: disabled.
-
+- `--export-training-curves`: export training curves as PNG images after every epoch. They are saved in the checkpoint
+  directory. Default: disabled.
 
-If `--no-eval` is not specified, the model is tested at the end of each epoch, although the reported metric is merely an approximation of the final result (for performance reasons). Once training is over, the model is automatically tested using the full procedure. This means that you can also specify the testing parameters when training.
+If `--no-eval` is not specified, the model is tested at the end of each epoch, although the reported metric is merely an
+approximation of the final result (for performance reasons). Once training is over, the model is automatically tested
+using the full procedure. This means that you can also specify the testing parameters when training.
 
 Here is a description of the model hyperparameters:
-- `-s` or `--stride`: the chunk size used for training, i.e. the number of frames that are predicted at once from each sequence. Increasing this value improves training speed at the expense of the error (due to correlated batch statistics). Default: `1` frame, which ensures maximum decorrelation. When this value is set to `1`, we also employ an optimized implementation of the model (see implementation details).
-- `-b` or `--batch-size`: the batch size used for training the model, in terms of *output frames* (regardless of the stride/chunk length). Default: `1024` frames.
+
+- `-s` or `--stride`: the chunk size used for training, i.e. the number of frames that are predicted at once from each
+  sequence. Increasing this value improves training speed at the expense of the error (due to correlated batch
+  statistics). Default: `1` frame, which ensures maximum decorrelation. When this value is set to `1`, we also employ an
+  optimized implementation of the model (see implementation details).
+- `-b` or `--batch-size`: the batch size used for training the model, in terms of *output frames* (regardless of the
+  stride/chunk length). Default: `1024` frames.
 - `-drop` or `--dropout`: dropout probability. Default: `0.25`.
 - `-lr` or `--learning-rate`: initial learning rate. Default: `0.001`.
 - `-lrd` or `--lr-decay`: learning rate decay after every epoch (multiplicative coefficient). Default: `0.95`.
-- `-no-tta` or `--no-test-time-augmentation`: disable test-time augmentation (which is enabled by default), i.e. do not flip poses horizontally when testing the model. Only effective when combined with data augmentation, so if you disable this you should also disable train-time data augmentation.
-- `-no-da` or `--no-data-augmentation`: disable train-time data augmentation (which is enabled by default), i.e. do not flip poses horizontally to double the training data.
-- `-arc` or `--architecture`: filter widths (only odd numbers supported) separated by commas. This parameter also specifies the number of residual blocks, and determines the receptive field of the model. The first number refers to the input layer, and is followed by the filter widths of the residual blocks. For instance, `3,5,5` uses `3x1` convolutions in the first layer, followed by two residual blocks with `5x1` convolutions. Default: `3,3,3`. Some valid examples are:
--- `-arc 3,3,3` (27 frames)
--- `-arc 3,3,7` (63 frames)
--- `-arc 3,3,3,3` (81 frames)
--- `-arc 3,3,3,3,3` (243 frames)
-- `--causal`: use causal (i.e. asymmetric) convolutions instead of symmetric convolutions. Causal convolutions are suitable for real-time applications because they do not exploit future frames (they only look in the past), but symmetric convolutions result in a better error since they can consider both past and future data. See below for more details.  Default: disabled.
+- `-no-tta` or `--no-test-time-augmentation`: disable test-time augmentation (which is enabled by default), i.e. do not
+  flip poses horizontally when testing the model. Only effective when combined with data augmentation, so if you disable
+  this you should also disable train-time data augmentation.
+- `-no-da` or `--no-data-augmentation`: disable train-time data augmentation (which is enabled by default), i.e. do not
+  flip poses horizontally to double the training data.
+- `-arc` or `--architecture`: filter widths (only odd numbers supported) separated by commas. This parameter also
+  specifies the number of residual blocks, and determines the receptive field of the model. The first number refers to
+  the input layer, and is followed by the filter widths of the residual blocks. For instance, `3,5,5` uses `3x1`
+  convolutions in the first layer, followed by two residual blocks with `5x1` convolutions. Default: `3,3,3`. Some valid
+  examples are:
+  -- `-arc 3,3,3` (27 frames)
+  -- `-arc 3,3,7` (63 frames)
+  -- `-arc 3,3,3,3` (81 frames)
+  -- `-arc 3,3,3,3,3` (243 frames)
+- `--causal`: use causal (i.e. asymmetric) convolutions instead of symmetric convolutions. Causal convolutions are
+  suitable for real-time applications because they do not exploit future frames (they only look in the past), but
+  symmetric convolutions result in a better error since they can consider both past and future data. See below for more
+  details. Default: disabled.
 - `-ch` or `--channels`: number of channels in convolutions. Default: `1024`.
-- `--dense`: use dense convolutions instead of dilated convolutions. This is only useful for benchmarks and ablation experiments.
-- `--disable-optimizations`: disable the optimized implementation when `--stride` == `1`. This is only useful for benchmarks.
+- `--dense`: use dense convolutions instead of dilated convolutions. This is only useful for benchmarks and ablation
+  experiments.
+- `--disable-optimizations`: disable the optimized implementation when `--stride` == `1`. This is only useful for
+  benchmarks.
 
 ## Semi-supervised training
+
 Semi-supervised learning is only implemented for Human3.6M.
 
-- `-sun` or `--subjects-unlabeled`: specifies the list of unlabeled subjects that are used for semi-supervision (separated by commas). Semi-supervised learning is automatically enabled when this parameter is set.
-- `--warmup`: number of supervised training epochs before attaching the semi-supervised loss. Default: `1` epoch. You may want to increase this when downsampling the dataset.
-- `--subset`: reduce the size of the training set by a given factor (a real number). E.g. `0.1` uses one tenth of the training data. Subsampling is achieved by extracting a random contiguous chunk from each video, while preserving the original frame rate. Default: `1` (i.e. disabled). This parameter can also be used in a supervised setting, but it is especially useful to simulate data scarcity in a semi-supervised setting.
+- `-sun` or `--subjects-unlabeled`: specifies the list of unlabeled subjects that are used for semi-supervision (
+  separated by commas). Semi-supervised learning is automatically enabled when this parameter is set.
+- `--warmup`: number of supervised training epochs before attaching the semi-supervised loss. Default: `1` epoch. You
+  may want to increase this when downsampling the dataset.
+- `--subset`: reduce the size of the training set by a given factor (a real number). E.g. `0.1` uses one tenth of the
+  training data. Subsampling is achieved by extracting a random contiguous chunk from each video, while preserving the
+  original frame rate. Default: `1` (i.e. disabled). This parameter can also be used in a supervised setting, but it is
+  especially useful to simulate data scarcity in a semi-supervised setting.
 - `--downsample`: reduce the dataset frame rate by an integer factor. Default: `1` (i.e. disabled).
-- `--no-bone-length`: do not add the bone length term to the unsupervised loss function (only useful for ablation experiments).
-- `--linear-projection`: ignore non-linear camera distortion parameters when performing projection to 2D, i.e. use only focal length and principal point.
+- `--no-bone-length`: do not add the bone length term to the unsupervised loss function (only useful for ablation
+  experiments).
+- `--linear-projection`: ignore non-linear camera distortion parameters when performing projection to 2D, i.e. use only
+  focal length and principal point.
 - `--no-proj`: do not add the projection consistency term to the loss function (only useful for ablations).
 
 ## Testing
-To test a particular model, you need to specify the checkpoint file via the `--evaluate` parameter, which will be loaded from the checkpoint directory (default: `checkpoint/`, but you can change it using the `-c` parameter). You also need to specify the same settings/hyperparameters that you used for training (e.g. input keypoints, architecture, etc.). The script will not run any compatibility checks -- this is a design choice to facilitate ablation experiments.
+
+To test a particular model, you need to specify the checkpoint file via the `--evaluate` parameter, which will be loaded
+from the checkpoint directory (default: `checkpoint/`, but you can change it using the `-c` parameter). You also need to
+specify the same settings/hyperparameters that you used for training (e.g. input keypoints, architecture, etc.). The
+script will not run any compatibility checks -- this is a design choice to facilitate ablation experiments.
 
 ## Visualization
-You can render videos by specifying both `--evaluate` and  `--render`. The script generates a visualization which contains three viewports: the 2D input keypoints (and optionally, a video overlay), the 3D reconstruction, and the 3D ground truth.
-Note that when you specify a video, the 2D detections are still loaded from the dataset according to the given parameters. It is up to you to choose the correct video. You can also visualize unlabeled videos -- in this case, the ground truth will not be shown.
+
+You can render videos by specifying both `--evaluate` and  `--render`. The script generates a visualization which
+contains three viewports: the 2D input keypoints (and optionally, a video overlay), the 3D reconstruction, and the 3D
+ground truth.
+Note that when you specify a video, the 2D detections are still loaded from the dataset according to the given
+parameters. It is up to you to choose the correct video. You can also visualize unlabeled videos -- in this case, the
+ground truth will not be shown.
 
 Here is a list of the command-line arguments related to visualization:
+
 - `--viz-subject`: subject to render, e.g. `S1`.
 - `--viz-action`: action to render, e.g. `Walking` or `Walking 1`.
 - `--viz-camera`: camera to render (integer), from 0 to 3 for Human3.6M, 0 to 2 for HumanEva. Default: `0`.
-- `--viz-video`: path to the 2D video to show. If specified, the script will render a skeleton overlay on top of the video. If not specified, a black background will be rendered instead (but the 2D detections will still be shown). 
+- `--viz-video`: path to the 2D video to show. If specified, the script will render a skeleton overlay on top of the
+  video. If not specified, a black background will be rendered instead (but the 2D detections will still be shown).
 - `--viz-skip`: skip the first N frames from the specified video. Useful for HumanEva. Default: `0`.
 - `--viz-output`: output file name (either a `.mp4` or `.gif` file).
 - `--viz-bitrate`: bitrate for MP4 videos. Default: `3000`.
-- `--viz-no-ground-truth`: by default, the videos contain three viewports: the 2D input pose, the 3D reconstruction, and the 3D ground truth. This flags removes the last one.
+- `--viz-no-ground-truth`: by default, the videos contain three viewports: the 2D input pose, the 3D reconstruction, and
+  the 3D ground truth. This flags removes the last one.
 - `--viz-limit`: render only first N frames. By default, all frames are rendered.
-- `--viz-downsample`: downsample videos by the specified factor, i.e. reduce the frame rate. E.g. if set to `2`, the frame rate is reduced from 50 FPS to 25 FPS. Default: `1` (no downsampling).
+- `--viz-downsample`: downsample videos by the specified factor, i.e. reduce the frame rate. E.g. if set to `2`, the
+  frame rate is reduced from 50 FPS to 25 FPS. Default: `1` (no downsampling).
 - `--viz-size`: output resolution multiplier. Higher = larger images. Default: `5`.
 - `--viz-export`: export 3D joint coordinates (in camera space) to the specified NumPy archive.
 
 Example:
+
 ```
 python run.py -k cpn_ft_h36m_dbb -arc 3,3,3,3,3 -c checkpoint --evaluate pretrained_h36m_cpn.bin --render --viz-subject S11 --viz-action Walking --viz-camera 0 --viz-video "/path/to/videos/S11/Videos/Walking.54138969.mp4" --viz-output output.gif --viz-size 3 --viz-downsample 2 --viz-limit 60
 ```
+
 ![](images/demo_h36m.gif)
 
-Generates a visualization for S11/Walking from camera 0, and exports the first frames to a GIF animation with a frame rate of 25 FPS. If you remove the `--viz-video` parameter, the skeleton overlay will be rendered on a blank background.
+Generates a visualization for S11/Walking from camera 0, and exports the first frames to a GIF animation with a frame
+rate of 25 FPS. If you remove the `--viz-video` parameter, the skeleton overlay will be rendered on a blank background.
 
-While Human3.6M visualization works out of the box, HumanEva visualization is trickier because the original videos must be segmented manually. Additionally, invalid frames and software synchronization complicate matters. Nonetheless, you can get decent visualizations by selecting the chunk 0 of validation sequences (which start at the beginning of each video) and discarding the first frames using `--viz-skip`. For a suggestion on the number of frames to skip, take a look at `sync_data` in `data/prepare_data_humaneva.py`.
+While Human3.6M visualization works out of the box, HumanEva visualization is trickier because the original videos must
+be segmented manually. Additionally, invalid frames and software synchronization complicate matters. Nonetheless, you
+can get decent visualizations by selecting the chunk 0 of validation sequences (which start at the beginning of each
+video) and discarding the first frames using `--viz-skip`. For a suggestion on the number of frames to skip, take a look
+at `sync_data` in `data/prepare_data_humaneva.py`.
 
 Example:
+
 ```
 python run.py -d humaneva15 -k detectron_pt_coco -str Train/S1,Train/S2,Train/S3 -ste Validate/S1,Validate/S2,Validate/S3 -c checkpoint --evaluate pretrained_humaneva15_detectron.bin  --render --viz-subject Validate/S2 --viz-action "Walking 1 chunk0" --viz-camera 0 --viz-output output_he.gif --viz-size 3 --viz-downsample 2 --viz-video "/path/to/videos/S2/Walking_1_(C1).avi" --viz-skip 115 --viz-limit 60
 ```
+
 ![](images/demo_humaneva.gif)
 
-Unlabeled videos are easier to visualize because they do not require synchronization with the ground truth. In this case, visualization works out of the box even for HumanEva.
+Unlabeled videos are easier to visualize because they do not require synchronization with the ground truth. In this
+case, visualization works out of the box even for HumanEva.
 
 Example:
+
 ```
 python run.py -d humaneva15 -k detectron_pt_coco -str Train/S1,Train/S2,Train/S3 -ste Validate/S1,Validate/S2,Validate/S3 -c checkpoint --evaluate pretrained_humaneva15_detectron.bin  --render --viz-subject Unlabeled/S4 --viz-action "Box 2" --viz-camera 0 --viz-output output_he.gif --viz-size 3 --viz-downsample 2 --viz-video "/path/to/videos/S4/Box_2_(C1).avi" --viz-limit 60
 ```
+
 ![](images/demo_humaneva_unlabeled.gif)
 
 ## Implementation details
+
 ### Batch generation during training
+
 Some details of our training procedure are better understood visually.
 ![](images/batching.png)
-The figure above shows how training batches are generated, depending on the value of `--stride` (from left to right: 1, 2, and 4). This example shows a sequence of 2D poses which has a length of N = 8 frames. The 3D poses (blue boxes in the figure) are inferred using a model that has a receptive field F = 5 frames. Therefore, because of valid padding, an input sequence of length N results in an output sequence of length N - F + 1, i.e. N - 4 in this example.
-
-When `--stride=1`, we generate one training example for each frame. This ensures that the batches are maximally uncorrelated, which helps batch normalization as well as generalization. As `--stride` increases, training becomes faster because the model can reutilize intermediate computations, at the cost of biased batch statistics. However, we provide an optimized implementation when `--stride=1`, which replaces dilated convolutions with strided convolutions (only while training), so in principle you should not touch this parameter unless you want to run specific experiments. To understand how it works, see the figures below:
+The figure above shows how training batches are generated, depending on the value of `--stride` (from left to right: 1,
+2, and 4). This example shows a sequence of 2D poses which has a length of N = 8 frames. The 3D poses (blue boxes in the
+figure) are inferred using a model that has a receptive field F = 5 frames. Therefore, because of valid padding, an
+input sequence of length N results in an output sequence of length N - F + 1, i.e. N - 4 in this example.
+
+When `--stride=1`, we generate one training example for each frame. This ensures that the batches are maximally
+uncorrelated, which helps batch normalization as well as generalization. As `--stride` increases, training becomes
+faster because the model can reutilize intermediate computations, at the cost of biased batch statistics. However, we
+provide an optimized implementation when `--stride=1`, which replaces dilated convolutions with strided convolutions (
+only while training), so in principle you should not touch this parameter unless you want to run specific experiments.
+To understand how it works, see the figures below:
 
 ![](images/convolutions_1f_naive.png)
-The figure above shows the information flow for a model with a receptive field of 27 frames, and a single-frame prediction, i.e. from N = 27 input frames we end up with one output frame. You can observe that this regular implementation tends to waste some intermediate results when a small number of frames are predicted. However, for inference of very long sequences, this approach is very efficient as intermediate results are shared among successive frames.
+The figure above shows the information flow for a model with a receptive field of 27 frames, and a single-frame
+prediction, i.e. from N = 27 input frames we end up with one output frame. You can observe that this regular
+implementation tends to waste some intermediate results when a small number of frames are predicted. However, for
+inference of very long sequences, this approach is very efficient as intermediate results are shared among successive
+frames.
 
 ![](images/convolutions_1f_optimized.png)
-Therefore, for training *only*, we use the implementation above, which replaces dilated convolutions with strided convolutions. It achieves the same result, but avoids computing unnecessary intermediate results.
+Therefore, for training *only*, we use the implementation above, which replaces dilated convolutions with strided
+convolutions. It achieves the same result, but avoids computing unnecessary intermediate results.
 
 ### Symmetric convolutions vs causal convolutions
-The figures below show the information flow from input (bottom) to output (top). In this example, we adopt a model with a receptive field of 27 frames.
+
+The figures below show the information flow from input (bottom) to output (top). In this example, we adopt a model with
+a receptive field of 27 frames.
 
 ![](images/convolutions_normal.png)
 With symmetric convolutions, both past and future information is exploited, resulting in a better reconstruction.
 
 ![](images/convolutions_causal.png)
-With causal convolutions, only past data is exploited. This approach is suited to real-time applications where future data cannot be exploited, at the cost of a slightly higher error.
+With causal convolutions, only past data is exploited. This approach is suited to real-time applications where future
+data cannot be exploited, at the cost of a slightly higher error.

backend/video_pose/INFERENCE.md

@@ -2,27 +2,47 @@
 
 **Update:** we have added support for Detectron2.
 
-In this short tutorial, we show how to run our model on arbitrary videos and visualize the predictions. Note that this feature is only provided for experimentation/research purposes and presents some limitations, as this repository is meant to provide a reference implementation of the approach described in the paper (not production-ready code for inference in the wild).
+In this short tutorial, we show how to run our model on arbitrary videos and visualize the predictions. Note that this
+feature is only provided for experimentation/research purposes and presents some limitations, as this repository is
+meant to provide a reference implementation of the approach described in the paper (not production-ready code for
+inference in the wild).
 
-Our script assumes that a video depicts *exactly* one person. In case of multiple people visible at once, the script will select the person corresponding to the bounding box with the highest confidence, which may cause glitches.
+Our script assumes that a video depicts *exactly* one person. In case of multiple people visible at once, the script
+will select the person corresponding to the bounding box with the highest confidence, which may cause glitches.
 
-The instructions below show how to use Detectron to infer 2D keypoints from videos, convert them to a custom dataset for our code, and infer 3D poses. For now, we do not have instructions for CPN. In the last section of this tutorial, we also provide some tips.
+The instructions below show how to use Detectron to infer 2D keypoints from videos, convert them to a custom dataset for
+our code, and infer 3D poses. For now, we do not have instructions for CPN. In the last section of this tutorial, we
+also provide some tips.
 
 ## Step 1: setup
+
 The inference script requires `ffmpeg`, which you can easily install via conda, pip, or manually.
 
-Download the [pretrained model](https://dl.fbaipublicfiles.com/video-pose-3d/pretrained_h36m_detectron_coco.bin) for generating 3D predictions. This model is different than the pretrained ones listed in the main README, as it expects input keypoints in COCO format (generated by the pretrained Detectron model) and outputs 3D joint positions in Human3.6M format. Put this model in the `checkpoint` directory of this repo.
+Download the [pretrained model](https://dl.fbaipublicfiles.com/video-pose-3d/pretrained_h36m_detectron_coco.bin) for
+generating 3D predictions. This model is different than the pretrained ones listed in the main README, as it expects
+input keypoints in COCO format (generated by the pretrained Detectron model) and outputs 3D joint positions in Human3.6M
+format. Put this model in the `checkpoint` directory of this repo.
 
-**Note:** if you had downloaded `d-pt-243.bin`, you should download the new pretrained model using the link above. `d-pt-243.bin` takes the keypoint probabilities as input (in addition to the x, y coordinates), which causes problems on videos with a different resolution than that of Human3.6M. The new model is only trained on 2D coordinates and works with any resolution/aspect ratio.
+**Note:** if you had downloaded `d-pt-243.bin`, you should download the new pretrained model using the link
+above. `d-pt-243.bin` takes the keypoint probabilities as input (in addition to the x, y coordinates), which causes
+problems on videos with a different resolution than that of Human3.6M. The new model is only trained on 2D coordinates
+and works with any resolution/aspect ratio.
 
 ## Step 2 (optional): video preprocessing
-Since the script expects a single-person scenario, you may want to extract a portion of your video. This is very easy to do with ffmpeg, e.g.
+
+Since the script expects a single-person scenario, you may want to extract a portion of your video. This is very easy to
+do with ffmpeg, e.g.
+
 ```
 ffmpeg -i input.mp4 -ss 1:00 -to 1:30 -c copy output.mp4
 ```
+
 extracts a clip from minute 1:00 to minute 1:30 of `input.mp4`, and exports it to `output.mp4`.
 
-Optionally, you can also adapt the frame rate of the video. Most videos have a frame rate of about 25 FPS, but our Human3.6M model was trained on 50-FPS videos. Since our model is robust to alterations in speed, this step is not very important and can be skipped, but if you want the best possible results you can use ffmpeg again for this task:
+Optionally, you can also adapt the frame rate of the video. Most videos have a frame rate of about 25 FPS, but our
+Human3.6M model was trained on 50-FPS videos. Since our model is robust to alterations in speed, this step is not very
+important and can be skipped, but if you want the best possible results you can use ffmpeg again for this task:
+
 ```
 ffmpeg -i input.mp4 -filter "minterpolate='fps=50'" -crf 0 output.mp4
 ```
@@ -30,9 +50,13 @@ ffmpeg -i input.mp4 -filter "minterpolate='fps=50'" -crf 0 output.mp4
 ## Step 3: inferring 2D keypoints with Detectron
 
 ### Using Detectron2 (new)
-Set up [Detectron2](https://github.com/facebookresearch/detectron2) and use the script  `inference/infer_video_d2.py` (no need to copy this, as it directly uses the Detectron2 API). This script provides a convenient interface to generate 2D keypoint predictions from videos without manually extracting individual frames.
+
+Set up [Detectron2](https://github.com/facebookresearch/detectron2) and use the script  `inference/infer_video_d2.py` (
+no need to copy this, as it directly uses the Detectron2 API). This script provides a convenient interface to generate
+2D keypoint predictions from videos without manually extracting individual frames.
 
 To infer keypoints from all the mp4 videos in `input_directory`, run
+
 ```
 cd inference
 python infer_video_d2.py \
@@ -41,16 +65,25 @@ python infer_video_d2.py \
     --image-ext mp4 \
     input_directory
 ```
-The results will be exported to `output_directory` as custom NumPy archives (`.npz` files). You can change the video extension in `--image-ext` (ffmpeg supports a wide range of formats).
 
-**Note:** although the architecture is the same (ResNet-101), the weights used by the Detectron2 model are not the same as those used by Detectron1. Since our pretrained model was trained on Detectron1 poses, the result might be slightly different (but it should still be pretty close).
+The results will be exported to `output_directory` as custom NumPy archives (`.npz` files). You can change the video
+extension in `--image-ext` (ffmpeg supports a wide range of formats).
+
+**Note:** although the architecture is the same (ResNet-101), the weights used by the Detectron2 model are not the same
+as those used by Detectron1. Since our pretrained model was trained on Detectron1 poses, the result might be slightly
+different (but it should still be pretty close).
 
 ### Using Detectron1 (old instructions)
-Set up [Detectron](https://github.com/facebookresearch/Detectron) and copy the script `inference/infer_video.py` from this repo to the `tools` directory of the Detectron repo. This script provides a convenient interface to generate 2D keypoint predictions from videos without manually extracting individual frames.
 
-Our Detectron script `infer_video.py` is a simple adaptation of `infer_simple.py` (which works on images) and has a similar command-line syntax.
+Set up [Detectron](https://github.com/facebookresearch/Detectron) and copy the script `inference/infer_video.py` from
+this repo to the `tools` directory of the Detectron repo. This script provides a convenient interface to generate 2D
+keypoint predictions from videos without manually extracting individual frames.
+
+Our Detectron script `infer_video.py` is a simple adaptation of `infer_simple.py` (which works on images) and has a
+similar command-line syntax.
 
 To infer keypoints from all the mp4 videos in `input_directory`, run
+
 ```
 python tools/infer_video.py \
     --cfg configs/12_2017_baselines/e2e_keypoint_rcnn_R-101-FPN_s1x.yaml \
@@ -59,27 +92,50 @@ python tools/infer_video.py \
 	--wts https://dl.fbaipublicfiles.com/detectron/37698009/12_2017_baselines/e2e_keypoint_rcnn_R-101-FPN_s1x.yaml.08_45_57.YkrJgP6O/output/train/keypoints_coco_2014_train:keypoints_coco_2014_valminusminival/generalized_rcnn/model_final.pkl \
     input_directory
 ```
-The results will be exported to `output_directory` as custom NumPy archives (`.npz` files). You can change the video extension in `--image-ext` (ffmpeg supports a wide range of formats).
+
+The results will be exported to `output_directory` as custom NumPy archives (`.npz` files). You can change the video
+extension in `--image-ext` (ffmpeg supports a wide range of formats).
 
 ## Step 4: creating a custom dataset
+
 Run our dataset preprocessing script from the `data` directory:
+
 ```
 python prepare_data_2d_custom.py -i /path/to/detections/output_directory -o myvideos
 ```
-This creates a custom dataset named `myvideos` (which contains all the videos in `output_directory`, each of which is mapped to a different subject) and saved to `data_2d_custom_myvideos.npz`. You are free to specify any name for the dataset.
 
-**Note:** as mentioned, the script will take the bounding box with the highest probability for each frame. If a particular frame has no bounding boxes, it is assumed to be a missed detection and the keypoints will be interpolated from neighboring frames.
+This creates a custom dataset named `myvideos` (which contains all the videos in `output_directory`, each of which is
+mapped to a different subject) and saved to `data_2d_custom_myvideos.npz`. You are free to specify any name for the
+dataset.
+
+**Note:** as mentioned, the script will take the bounding box with the highest probability for each frame. If a
+particular frame has no bounding boxes, it is assumed to be a missed detection and the keypoints will be interpolated
+from neighboring frames.
 
 ## Step 5: rendering a custom video and exporting coordinates
-You can finally use the visualization feature to render a video of the 3D joint predictions. You must specify the `custom` dataset (`-d custom`), the input keypoints as exported in the previous step (`-k myvideos`), the correct architecture/checkpoint, and the action `custom` (`--viz-action custom`). The subject is the file name of the input video, and the camera is always 0.
+
+You can finally use the visualization feature to render a video of the 3D joint predictions. You must specify
+the `custom` dataset (`-d custom`), the input keypoints as exported in the previous step (`-k myvideos`), the correct
+architecture/checkpoint, and the action `custom` (`--viz-action custom`). The subject is the file name of the input
+video, and the camera is always 0.
+
 ```
 python run.py -d custom -k myvideos -arc 3,3,3,3,3 -c checkpoint --evaluate pretrained_h36m_detectron_coco.bin --render --viz-subject input_video.mp4 --viz-action custom --viz-camera 0 --viz-video /path/to/input_video.mp4 --viz-output output.mp4 --viz-size 6
 ```
 
-You can also export the 3D joint positions (in camera space) to a NumPy archive. To this end, replace `--viz-output` with `--viz-export` and specify the file name.
+You can also export the 3D joint positions (in camera space) to a NumPy archive. To this end, replace `--viz-output`
+with `--viz-export` and specify the file name.
 
 ## Limitations and tips
-- The model was trained on Human3.6M cameras (which are relatively undistorted), and the results may be bad if the intrinsic parameters of the cameras of your videos differ much from those of Human3.6M. This may be particularly noticeable with fisheye cameras, which present a high degree of non-linear lens distortion. If the camera parameters are known, consider preprocessing your videos to match those of Human3.6M as closely as possible.
-- If you want multi-person tracking, you should implement a bounding box matching strategy. An example would be to use bipartite matching on the bounding box overlap (IoU) between subsequent frames, but there are many other approaches.
-- Predictions are relative to the root joint, i.e. the global trajectory is not regressed. If you need it, you may want to use another model to regress it, such as the one we use for semi-supervision.
-- Predictions are always in *camera space* (regardless of whether the trajectory is available). For our visualization script, we simply take a random camera from Human3.6M, which fits decently most videos where the camera viewport is parallel to the ground. 
+
+- The model was trained on Human3.6M cameras (which are relatively undistorted), and the results may be bad if the
+  intrinsic parameters of the cameras of your videos differ much from those of Human3.6M. This may be particularly
+  noticeable with fisheye cameras, which present a high degree of non-linear lens distortion. If the camera parameters
+  are known, consider preprocessing your videos to match those of Human3.6M as closely as possible.
+- If you want multi-person tracking, you should implement a bounding box matching strategy. An example would be to use
+  bipartite matching on the bounding box overlap (IoU) between subsequent frames, but there are many other approaches.
+- Predictions are relative to the root joint, i.e. the global trajectory is not regressed. If you need it, you may want
+  to use another model to regress it, such as the one we use for semi-supervision.
+- Predictions are always in *camera space* (regardless of whether the trajectory is available). For our visualization
+  script, we simply take a random camera from Human3.6M, which fits decently most videos where the camera viewport is
+  parallel to the ground. 

backend/video_pose/README.md

@@ -1,8 +1,11 @@
 # 3D human pose estimation in video with temporal convolutions and semi-supervised training
+
 <p align="center"><img src="images/convolutions_anim.gif" width="50%" alt="" /></p>
 
 This is the implementation of the approach described in the paper:
-> Dario Pavllo, Christoph Feichtenhofer, David Grangier, and Michael Auli. [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://arxiv.org/abs/1811.11742). In Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
+> Dario Pavllo, Christoph Feichtenhofer, David Grangier, and Michael
+> Auli. [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://arxiv.org/abs/1811.11742)
+> . In Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
 
 More demos are available at https://dariopavllo.github.io/VideoPose3D
 
@@ -11,6 +14,7 @@ More demos are available at https://dariopavllo.github.io/VideoPose3D
 ![](images/demo_temporal.gif)
 
 ### Results on Human3.6M
+
 Under Protocol 1 (mean per-joint position error) and Protocol 2 (mean-per-joint position error after rigid alignment).
 
 | 2D Detections | BBoxes | Blocks | Receptive Field | Error (P1) | Error (P2) |
@@ -23,24 +27,37 @@ Under Protocol 1 (mean per-joint position error) and Protocol 2 (mean-per-joint
 | Ground truth | -- | 4 | 243 frames | 37.2 mm | 27.2 mm |
 
 ## Quick start
-To get started as quickly as possible, follow the instructions in this section. This should allow you train a model from scratch, test our pretrained models, and produce basic visualizations. For more detailed instructions, please refer to [`DOCUMENTATION.md`](DOCUMENTATION.md).
+
+To get started as quickly as possible, follow the instructions in this section. This should allow you train a model from
+scratch, test our pretrained models, and produce basic visualizations. For more detailed instructions, please refer
+to [`DOCUMENTATION.md`](DOCUMENTATION.md).
 
 ### Dependencies
+
 Make sure you have the following dependencies installed before proceeding:
+
 - Python 3+ distribution
 - PyTorch >= 0.4.0
 
 Optional:
-- Matplotlib, if you want to visualize predictions. Additionally, you need *ffmpeg* to export MP4 videos, and *imagemagick* to export GIFs.
-- MATLAB, if you want to experiment with HumanEva-I (you need this to convert the dataset). 
+
+- Matplotlib, if you want to visualize predictions. Additionally, you need *ffmpeg* to export MP4 videos, and *
+  imagemagick* to export GIFs.
+- MATLAB, if you want to experiment with HumanEva-I (you need this to convert the dataset).
 
 ### Dataset setup
-You can find the instructions for setting up the Human3.6M and HumanEva-I datasets in [`DATASETS.md`](DATASETS.md). For this short guide, we focus on Human3.6M. You are not required to setup HumanEva, unless you want to experiment with it.
+
+You can find the instructions for setting up the Human3.6M and HumanEva-I datasets in [`DATASETS.md`](DATASETS.md). For
+this short guide, we focus on Human3.6M. You are not required to setup HumanEva, unless you want to experiment with it.
 
 In order to proceed, you must also copy CPN detections (for Human3.6M) and/or Mask R-CNN detections (for HumanEva).
 
 ### Evaluating our pretrained models
-The pretrained models can be downloaded from AWS. Put `pretrained_h36m_cpn.bin` (for Human3.6M) and/or `pretrained_humaneva15_detectron.bin` (for HumanEva) in the `checkpoint/` directory (create it if it does not exist).
+
+The pretrained models can be downloaded from AWS. Put `pretrained_h36m_cpn.bin` (for Human3.6M)
+and/or `pretrained_humaneva15_detectron.bin` (for HumanEva) in the `checkpoint/` directory (create it if it does not
+exist).
+
 ```sh
 mkdir checkpoint
 cd checkpoint
@@ -50,15 +67,20 @@ cd ..
 ```
 
 These models allow you to reproduce our top-performing baselines, which are:
-- 46.8 mm for Human3.6M, using fine-tuned CPN detections, bounding boxes from Mask R-CNN, and an architecture with a receptive field of 243 frames.
-- 33.0 mm for HumanEva-I (on 3 actions), using pretrained Mask R-CNN detections, and an architecture with a receptive field of 27 frames. This is the multi-action model trained on 3 actions (Walk, Jog, Box).
+
+- 46.8 mm for Human3.6M, using fine-tuned CPN detections, bounding boxes from Mask R-CNN, and an architecture with a
+  receptive field of 243 frames.
+- 33.0 mm for HumanEva-I (on 3 actions), using pretrained Mask R-CNN detections, and an architecture with a receptive
+  field of 27 frames. This is the multi-action model trained on 3 actions (Walk, Jog, Box).
 
 To test on Human3.6M, run:
+
 ```
 python run.py -k cpn_ft_h36m_dbb -arc 3,3,3,3,3 -c checkpoint --evaluate pretrained_h36m_cpn.bin
 ```
 
 To test on HumanEva, run:
+
 ```
 python run.py -d humaneva15 -k detectron_pt_coco -str Train/S1,Train/S2,Train/S3 -ste Validate/S1,Validate/S2,Validate/S3 -a Walk,Jog,Box --by-subject -c checkpoint --evaluate pretrained_humaneva15_detectron.bin
 ```
@@ -66,49 +88,73 @@ python run.py -d humaneva15 -k detectron_pt_coco -str Train/S1,Train/S2,Train/S3
 [`DOCUMENTATION.md`](DOCUMENTATION.md) provides a precise description of all command-line arguments.
 
 ### Inference in the wild
-We have introduced an experimental feature to run our model on custom videos. See [`INFERENCE.md`](INFERENCE.md) for more details.
+
+We have introduced an experimental feature to run our model on custom videos. See [`INFERENCE.md`](INFERENCE.md) for
+more details.
 
 ### Training from scratch
+
 If you want to reproduce the results of our pretrained models, run the following commands.
 
 For Human3.6M:
+
 ```
 python run.py -e 80 -k cpn_ft_h36m_dbb -arc 3,3,3,3,3
 ```
-By default the application runs in training mode. This will train a new model for 80 epochs, using fine-tuned CPN detections. Expect a training time of 24 hours on a high-end Pascal GPU. If you feel that this is too much, or your GPU is not powerful enough, you can train a model with a smaller receptive field, e.g.
-- `-arc 3,3,3,3` (81 frames) should require 11 hours and achieve 47.7 mm. 
+
+By default the application runs in training mode. This will train a new model for 80 epochs, using fine-tuned CPN
+detections. Expect a training time of 24 hours on a high-end Pascal GPU. If you feel that this is too much, or your GPU
+is not powerful enough, you can train a model with a smaller receptive field, e.g.
+
+- `-arc 3,3,3,3` (81 frames) should require 11 hours and achieve 47.7 mm.
 - `-arc 3,3,3` (27 frames) should require 6 hours and achieve 48.8 mm.
 
 You could also lower the number of epochs from 80 to 60 with a negligible impact on the result.
 
 For HumanEva:
+
 ```
 python run.py -d humaneva15 -k detectron_pt_coco -str Train/S1,Train/S2,Train/S3 -ste Validate/S1,Validate/S2,Validate/S3 -b 128 -e 1000 -lrd 0.996 -a Walk,Jog,Box --by-subject
 ```
+
 This will train for 1000 epochs, using Mask R-CNN detections and evaluating each subject separately.
 Since HumanEva is much smaller than Human3.6M, training should require about 50 minutes.
 
 ### Semi-supervised training
-To perform semi-supervised training, you just need to add the `--subjects-unlabeled` argument. In the example below, we use ground-truth 2D poses as input, and train supervised on just 10% of Subject 1 (specified by `--subset 0.1`). The remaining subjects are treated as unlabeled data and are used for semi-supervision.
+
+To perform semi-supervised training, you just need to add the `--subjects-unlabeled` argument. In the example below, we
+use ground-truth 2D poses as input, and train supervised on just 10% of Subject 1 (specified by `--subset 0.1`). The
+remaining subjects are treated as unlabeled data and are used for semi-supervision.
+
 ```
 python run.py -k gt --subjects-train S1 --subset 0.1 --subjects-unlabeled S5,S6,S7,S8 -e 200 -lrd 0.98 -arc 3,3,3 --warmup 5 -b 64
 ```
+
 This should give you an error around 65.2 mm. By contrast, if we only train supervised
+
 ```
 python run.py -k gt --subjects-train S1 --subset 0.1 -e 200 -lrd 0.98 -arc 3,3,3 -b 64
 ```
+
 we get around 80.7 mm, which is significantly higher.
 
 ### Visualization
+
 If you have the original Human3.6M videos, you can generate nice visualizations of the model predictions. For instance:
+
 ```
 python run.py -k cpn_ft_h36m_dbb -arc 3,3,3,3,3 -c checkpoint --evaluate pretrained_h36m_cpn.bin --render --viz-subject S11 --viz-action Walking --viz-camera 0 --viz-video "/path/to/videos/S11/Videos/Walking.54138969.mp4" --viz-output output.gif --viz-size 3 --viz-downsample 2 --viz-limit 60
 ```
-The script can also export MP4 videos, and supports a variety of parameters (e.g. downsampling/FPS, size, bitrate). See [`DOCUMENTATION.md`](DOCUMENTATION.md) for more details.
+
+The script can also export MP4 videos, and supports a variety of parameters (e.g. downsampling/FPS, size, bitrate).
+See [`DOCUMENTATION.md`](DOCUMENTATION.md) for more details.
 
 ## License
-This work is licensed under CC BY-NC. See LICENSE for details. Third-party datasets are subject to their respective licenses.
+
+This work is licensed under CC BY-NC. See LICENSE for details. Third-party datasets are subject to their respective
+licenses.
 If you use our code/models in your research, please cite our paper:
+
 ```
 @inproceedings{pavllo:videopose3d:2019,
   title={3D human pose estimation in video with temporal convolutions and semi-supervised training},

backend/video_pose/common/arguments.py

@@ -9,7 +9,6 @@ import argparse
 import os
 
 def parse_args(video_path, video_analyzed_path):
-
     video_name = os.path.basename(video_path)
 
     reconstructed_video_path = video_path.replace(".mp4", "_reconstructed.mp4")
@@ -79,12 +78,13 @@ def parse_args(video_path, video_analyzed_path):
     parser.add_argument('--no-proj', action='store_true', help='disable projection for semi-supervised setting')
 
     # Visualization
-    parser.add_argument('--viz-subject', type=str, default=video_name,metavar='STR', help='subject to render')
+    parser.add_argument('--viz-subject', type=str, default=video_name, metavar='STR', help='subject to render')
     parser.add_argument('--viz-action', type=str, default='custom', metavar='STR', help='action to render')
     parser.add_argument('--viz-camera', type=int, default=0, metavar='N', help='camera to render')
     parser.add_argument('--viz-video', type=str, default=video_path, metavar='PATH', help='path to input video')
     parser.add_argument('--viz-skip', type=int, default=0, metavar='N', help='skip first N frames of input video')
-    parser.add_argument('--viz-output', type=str, default=reconstructed_video_path, metavar='PATH', help='output file name (.gif or .mp4)')
+    parser.add_argument('--viz-output', type=str, default=reconstructed_video_path, metavar='PATH',
+                        help='output file name (.gif or .mp4)')
     parser.add_argument('--viz-export', type=str, metavar='PATH', help='output file name for coordinates')
     parser.add_argument('--viz-bitrate', type=int, default=3000, metavar='N', help='bitrate for mp4 videos')
     parser.add_argument('--viz-no-ground-truth', action='store_true', help='do not show ground-truth poses')

backend/video_pose/common/camera.py

@@ -8,32 +8,28 @@
 import numpy as np
 import torch
 
-from video_pose.common.utils import wrap
 from video_pose.common.quaternion import qrot, qinverse
+from video_pose.common.utils import wrap
 
-def normalize_screen_coordinates(X, w, h): 
+def normalize_screen_coordinates(X, w, h):
     assert X.shape[-1] == 2
-    
+
     # Normalize so that [0, w] is mapped to [-1, 1], while preserving the aspect ratio
-    return X/w*2 - [1, h/w]
+    return X / w * 2 - [1, h / w]
 
-    
 def image_coordinates(X, w, h):
     assert X.shape[-1] == 2
-    
+
     # Reverse camera frame normalization
-    return (X + [1, h/w])*w/2
-    
+    return (X + [1, h / w]) * w / 2
 
 def world_to_camera(X, R, t):
-    Rt = wrap(qinverse, R) # Invert rotation
-    return wrap(qrot, np.tile(Rt, (*X.shape[:-1], 1)), X - t) # Rotate and translate
+    Rt = wrap(qinverse, R)  # Invert rotation
+    return wrap(qrot, np.tile(Rt, (*X.shape[:-1], 1)), X - t)  # Rotate and translate
 
-    
 def camera_to_world(X, R, t):
     return wrap(qrot, np.tile(R, (*X.shape[:-1], 1)), X) + t
 
-    
 def project_to_2d(X, camera_params):
     """
     Project 3D points to 2D using the Human3.6M camera projection function.
@@ -47,24 +43,25 @@ def project_to_2d(X, camera_params):
     assert len(camera_params.shape) == 2
     assert camera_params.shape[-1] == 9
     assert X.shape[0] == camera_params.shape[0]
-    
+
     while len(camera_params.shape) < len(X.shape):
         camera_params = camera_params.unsqueeze(1)
-        
+
     f = camera_params[..., :2]
     c = camera_params[..., 2:4]
     k = camera_params[..., 4:7]
     p = camera_params[..., 7:]
-    
+
     XX = torch.clamp(X[..., :2] / X[..., 2:], min=-1, max=1)
-    r2 = torch.sum(XX[..., :2]**2, dim=len(XX.shape)-1, keepdim=True)
+    r2 = torch.sum(XX[..., :2] ** 2, dim=len(XX.shape) - 1, keepdim=True)
 
-    radial = 1 + torch.sum(k * torch.cat((r2, r2**2, r2**3), dim=len(r2.shape)-1), dim=len(r2.shape)-1, keepdim=True)
-    tan = torch.sum(p*XX, dim=len(XX.shape)-1, keepdim=True)
+    radial = 1 + torch.sum(k * torch.cat((r2, r2 ** 2, r2 ** 3), dim=len(r2.shape) - 1), dim=len(r2.shape) - 1,
+                           keepdim=True)
+    tan = torch.sum(p * XX, dim=len(XX.shape) - 1, keepdim=True)
 
-    XXX = XX*(radial + tan) + p*r2
-    
-    return f*XXX + c
+    XXX = XX * (radial + tan) + p * r2
+
+    return f * XXX + c
 
 def project_to_2d_linear(X, camera_params):
     """
@@ -78,13 +75,13 @@ def project_to_2d_linear(X, camera_params):
     assert len(camera_params.shape) == 2
     assert camera_params.shape[-1] == 9
     assert X.shape[0] == camera_params.shape[0]
-    
+
     while len(camera_params.shape) < len(X.shape):
         camera_params = camera_params.unsqueeze(1)
-        
+
     f = camera_params[..., :2]
     c = camera_params[..., 2:4]
-    
+
     XX = torch.clamp(X[..., :2] / X[..., 2:], min=-1, max=1)
-    
-    return f*XX + c
+
+    return f * XX + c

backend/video_pose/common/custom_dataset.py

@@ -5,33 +5,33 @@
 # LICENSE file in the root directory of this source tree.
 #
 
-import numpy as np
 import copy
-from video_pose.common.skeleton import Skeleton
-from video_pose.common.mocap_dataset import MocapDataset
+import numpy as np
+
 from video_pose.common.camera import normalize_screen_coordinates, image_coordinates
 from video_pose.common.h36m_dataset import h36m_skeleton
-       
+from video_pose.common.mocap_dataset import MocapDataset
+from video_pose.common.skeleton import Skeleton
 
 custom_camera_params = {
     'id': None,
-    'res_w': None, # Pulled from metadata
-    'res_h': None, # Pulled from metadata
-    
+    'res_w': None,  # Pulled from metadata
+    'res_h': None,  # Pulled from metadata
+
     # Dummy camera parameters (taken from Human3.6M), only for visualization purposes
-    'azimuth': 70, # Only used for visualization
+    'azimuth': 70,  # Only used for visualization
     'orientation': [0.1407056450843811, -0.1500701755285263, -0.755240797996521, 0.6223280429840088],
     'translation': [1841.1070556640625, 4955.28466796875, 1563.4454345703125],
 }
 
 class CustomDataset(MocapDataset):
     def __init__(self, detections_path, remove_static_joints=True):
-        super().__init__(fps=None, skeleton=h36m_skeleton)        
-        
+        super().__init__(fps=None, skeleton=h36m_skeleton)
+
         # Load serialized dataset
         data = np.load(detections_path, allow_pickle=True)
         resolutions = data['metadata'].item()['video_metadata']
-        
+
         self._cameras = {}
         self._data = {}
         for video_name, res in resolutions.items():
@@ -39,28 +39,27 @@ class CustomDataset(MocapDataset):
             cam.update(custom_camera_params)
             cam['orientation'] = np.array(cam['orientation'], dtype='float32')
             cam['translation'] = np.array(cam['translation'], dtype='float32')
-            cam['translation'] = cam['translation']/1000 # mm to meters
-            
+            cam['translation'] = cam['translation'] / 1000  # mm to meters
+
             cam['id'] = video_name
             cam['res_w'] = res['w']
             cam['res_h'] = res['h']
-            
+
             self._cameras[video_name] = [cam]
-        
+
             self._data[video_name] = {
                 'custom': {
                     'cameras': cam
                 }
             }
-                
+
         if remove_static_joints:
             # Bring the skeleton to 17 joints instead of the original 32
             self.remove_joints([4, 5, 9, 10, 11, 16, 20, 21, 22, 23, 24, 28, 29, 30, 31])
-            
+
             # Rewire shoulders to the correct parents
             self._skeleton._parents[11] = 8
             self._skeleton._parents[14] = 8
-            
+
     def supports_semi_supervised(self):
         return False
-   

backend/video_pose/common/generators.py

@@ -5,8 +5,8 @@
 # LICENSE file in the root directory of this source tree.
 #
 
-from itertools import zip_longest
 import numpy as np
+from itertools import zip_longest
 
 class ChunkedGenerator:
     """
@@ -27,6 +27,7 @@ class ChunkedGenerator:
     kps_left and kps_right -- list of left/right 2D keypoints if flipping is enabled
     joints_left and joints_right -- list of left/right 3D joints if flipping is enabled
     """
+
     def __init__(self, batch_size, cameras, poses_3d, poses_2d,
                  chunk_length, pad=0, causal_shift=0,
                  shuffle=True, random_seed=1234,
@@ -34,14 +35,14 @@ class ChunkedGenerator:
                  endless=False):
         assert poses_3d is None or len(poses_3d) == len(poses_2d), (len(poses_3d), len(poses_2d))
         assert cameras is None or len(cameras) == len(poses_2d)
-    
+
         # Build lineage info
-        pairs = [] # (seq_idx, start_frame, end_frame, flip) tuples
+        pairs = []  # (seq_idx, start_frame, end_frame, flip) tuples
         for i in range(len(poses_2d)):
             assert poses_3d is None or poses_3d[i].shape[0] == poses_3d[i].shape[0]
             n_chunks = (poses_2d[i].shape[0] + chunk_length - 1) // chunk_length
             offset = (n_chunks * chunk_length - poses_2d[i].shape[0]) // 2
-            bounds = np.arange(n_chunks+1)*chunk_length - offset
+            bounds = np.arange(n_chunks + 1) * chunk_length - offset
             augment_vector = np.full(len(bounds - 1), False, dtype=bool)
             pairs += zip(np.repeat(i, len(bounds - 1)), bounds[:-1], bounds[1:], augment_vector)
             if augment:
@@ -52,7 +53,7 @@ class ChunkedGenerator:
             self.batch_cam = np.empty((batch_size, cameras[0].shape[-1]))
         if poses_3d is not None:
             self.batch_3d = np.empty((batch_size, chunk_length, poses_3d[0].shape[-2], poses_3d[0].shape[-1]))
-        self.batch_2d = np.empty((batch_size, chunk_length + 2*pad, poses_2d[0].shape[-2], poses_2d[0].shape[-1]))
+        self.batch_2d = np.empty((batch_size, chunk_length + 2 * pad, poses_2d[0].shape[-2], poses_2d[0].shape[-1]))
 
         self.num_batches = (len(pairs) + batch_size - 1) // batch_size
         self.batch_size = batch_size
@@ -63,29 +64,29 @@ class ChunkedGenerator:
         self.causal_shift = causal_shift
         self.endless = endless
         self.state = None
-        
+
         self.cameras = cameras
         self.poses_3d = poses_3d
         self.poses_2d = poses_2d
-        
+
         self.augment = augment
         self.kps_left = kps_left
         self.kps_right = kps_right
         self.joints_left = joints_left
         self.joints_right = joints_right
-        
+
     def num_frames(self):
         return self.num_batches * self.batch_size
-    
+
     def random_state(self):
         return self.random
-    
+
     def set_random_state(self, random):
         self.random = random
-        
+
     def augment_enabled(self):
         return self.augment
-    
+
     def next_pairs(self):
         if self.state is None:
             if self.shuffle:
@@ -95,13 +96,13 @@ class ChunkedGenerator:
             return 0, pairs
         else:
             return self.state
-    
+
     def next_epoch(self):
         enabled = True
         while enabled:
             start_idx, pairs = self.next_pairs()
             for b_i in range(start_idx, self.num_batches):
-                chunks = pairs[b_i*self.batch_size : (b_i+1)*self.batch_size]
+                chunks = pairs[b_i * self.batch_size: (b_i + 1) * self.batch_size]
                 for i, (seq_i, start_3d, end_3d, flip) in enumerate(chunks):
                     start_2d = start_3d - self.pad - self.causal_shift
                     end_2d = end_3d + self.pad - self.causal_shift
@@ -113,14 +114,16 @@ class ChunkedGenerator:
                     pad_left_2d = low_2d - start_2d
                     pad_right_2d = end_2d - high_2d
                     if pad_left_2d != 0 or pad_right_2d != 0:
-                        self.batch_2d[i] = np.pad(seq_2d[low_2d:high_2d], ((pad_left_2d, pad_right_2d), (0, 0), (0, 0)), 'edge')
+                        self.batch_2d[i] = np.pad(seq_2d[low_2d:high_2d], ((pad_left_2d, pad_right_2d), (0, 0), (0, 0)),
+                                                  'edge')
                     else:
                         self.batch_2d[i] = seq_2d[low_2d:high_2d]
 
                     if flip:
                         # Flip 2D keypoints
                         self.batch_2d[i, :, :, 0] *= -1
-                        self.batch_2d[i, :, self.kps_left + self.kps_right] = self.batch_2d[i, :, self.kps_right + self.kps_left]
+                        self.batch_2d[i, :, self.kps_left + self.kps_right] = self.batch_2d[i, :,
+                                                                              self.kps_right + self.kps_left]
 
                     # 3D poses
                     if self.poses_3d is not None:
@@ -130,7 +133,8 @@ class ChunkedGenerator:
                         pad_left_3d = low_3d - start_3d
                         pad_right_3d = end_3d - high_3d
                         if pad_left_3d != 0 or pad_right_3d != 0:
-                            self.batch_3d[i] = np.pad(seq_3d[low_3d:high_3d], ((pad_left_3d, pad_right_3d), (0, 0), (0, 0)), 'edge')
+                            self.batch_3d[i] = np.pad(seq_3d[low_3d:high_3d],
+                                                      ((pad_left_3d, pad_right_3d), (0, 0), (0, 0)), 'edge')
                         else:
                             self.batch_3d[i] = seq_3d[low_3d:high_3d]
 
@@ -138,7 +142,7 @@ class ChunkedGenerator:
                             # Flip 3D joints
                             self.batch_3d[i, :, :, 0] *= -1
                             self.batch_3d[i, :, self.joints_left + self.joints_right] = \
-                                    self.batch_3d[i, :, self.joints_right + self.joints_left]
+                                self.batch_3d[i, :, self.joints_right + self.joints_left]
 
                     # Cameras
                     if self.cameras is not None:
@@ -158,12 +162,11 @@ class ChunkedGenerator:
                     yield self.batch_cam[:len(chunks)], None, self.batch_2d[:len(chunks)]
                 else:
                     yield self.batch_cam[:len(chunks)], self.batch_3d[:len(chunks)], self.batch_2d[:len(chunks)]
-            
+
             if self.endless:
                 self.state = None
             else:
                 enabled = False
-            
 
 class UnchunkedGenerator:
     """
@@ -183,7 +186,7 @@ class UnchunkedGenerator:
     kps_left and kps_right -- list of left/right 2D keypoints if flipping is enabled
     joints_left and joints_right -- list of left/right 3D joints if flipping is enabled
     """
-    
+
     def __init__(self, cameras, poses_3d, poses_2d, pad=0, causal_shift=0,
                  augment=False, kps_left=None, kps_right=None, joints_left=None, joints_right=None):
         assert poses_3d is None or len(poses_3d) == len(poses_2d)
@@ -194,46 +197,48 @@ class UnchunkedGenerator:
         self.kps_right = kps_right
         self.joints_left = joints_left
         self.joints_right = joints_right
-        
+
         self.pad = pad
         self.causal_shift = causal_shift
         self.cameras = [] if cameras is None else cameras
         self.poses_3d = [] if poses_3d is None else poses_3d
         self.poses_2d = poses_2d
-        
+
     def num_frames(self):
         count = 0
         for p in self.poses_2d:
             count += p.shape[0]
         return count
-    
+
     def augment_enabled(self):
         return self.augment
-    
+
     def set_augment(self, augment):
         self.augment = augment
-    
+
     def next_epoch(self):
         for seq_cam, seq_3d, seq_2d in zip_longest(self.cameras, self.poses_3d, self.poses_2d):
             batch_cam = None if seq_cam is None else np.expand_dims(seq_cam, axis=0)
             batch_3d = None if seq_3d is None else np.expand_dims(seq_3d, axis=0)
             batch_2d = np.expand_dims(np.pad(seq_2d,
-                            ((self.pad + self.causal_shift, self.pad - self.causal_shift), (0, 0), (0, 0)),
-                            'edge'), axis=0)
+                                             ((self.pad + self.causal_shift, self.pad - self.causal_shift), (0, 0),
+                                              (0, 0)),
+                                             'edge'), axis=0)
             if self.augment:
                 # Append flipped version
                 if batch_cam is not None:
                     batch_cam = np.concatenate((batch_cam, batch_cam), axis=0)
                     batch_cam[1, 2] *= -1
                     batch_cam[1, 7] *= -1
-                
+
                 if batch_3d is not None:
                     batch_3d = np.concatenate((batch_3d, batch_3d), axis=0)
                     batch_3d[1, :, :, 0] *= -1
-                    batch_3d[1, :, self.joints_left + self.joints_right] = batch_3d[1, :, self.joints_right + self.joints_left]
+                    batch_3d[1, :, self.joints_left + self.joints_right] = batch_3d[1, :,
+                                                                           self.joints_right + self.joints_left]
 
                 batch_2d = np.concatenate((batch_2d, batch_2d), axis=0)
                 batch_2d[1, :, :, 0] *= -1
                 batch_2d[1, :, self.kps_left + self.kps_right] = batch_2d[1, :, self.kps_right + self.kps_left]
 
-            yield batch_cam, batch_3d, batch_2d
+            yield batch_cam, batch_3d, batch_2d

backend/video_pose/common/h36m_dataset.py

@@ -5,16 +5,17 @@
 # LICENSE file in the root directory of this source tree.
 #
 
-import numpy as np
 import copy
-from video_pose.common.skeleton import Skeleton
-from video_pose.common.mocap_dataset import MocapDataset
+import numpy as np
+
 from video_pose.common.camera import normalize_screen_coordinates, image_coordinates
-       
-h36m_skeleton = Skeleton(parents=[-1,  0,  1,  2,  3,  4,  0,  6,  7,  8,  9,  0, 11, 12, 13, 14, 12,
-       16, 17, 18, 19, 20, 19, 22, 12, 24, 25, 26, 27, 28, 27, 30],
-       joints_left=[6, 7, 8, 9, 10, 16, 17, 18, 19, 20, 21, 22, 23],
-       joints_right=[1, 2, 3, 4, 5, 24, 25, 26, 27, 28, 29, 30, 31])
+from video_pose.common.mocap_dataset import MocapDataset
+from video_pose.common.skeleton import Skeleton
+
+h36m_skeleton = Skeleton(parents=[-1, 0, 1, 2, 3, 4, 0, 6, 7, 8, 9, 0, 11, 12, 13, 14, 12,
+                                  16, 17, 18, 19, 20, 19, 22, 12, 24, 25, 26, 27, 28, 27, 30],
+                         joints_left=[6, 7, 8, 9, 10, 16, 17, 18, 19, 20, 21, 22, 23],
+                         joints_right=[1, 2, 3, 4, 5, 24, 25, 26, 27, 28, 29, 30, 31])
 
 h36m_cameras_intrinsic_params = [
     {
@@ -25,7 +26,7 @@ h36m_cameras_intrinsic_params = [
         'tangential_distortion': [-0.0009756988729350269, -0.00142447161488235],
         'res_w': 1000,
         'res_h': 1002,
-        'azimuth': 70, # Only used for visualization
+        'azimuth': 70,  # Only used for visualization
     },
     {
         'id': '55011271',
@@ -35,7 +36,7 @@ h36m_cameras_intrinsic_params = [
         'tangential_distortion': [-0.0016190266469493508, -0.0027408944442868233],
         'res_w': 1000,
         'res_h': 1000,
-        'azimuth': -70, # Only used for visualization
+        'azimuth': -70,  # Only used for visualization
     },
     {
         'id': '58860488',
@@ -45,7 +46,7 @@ h36m_cameras_intrinsic_params = [
         'tangential_distortion': [0.0014843869721516967, -0.0007599993259645998],
         'res_w': 1000,
         'res_h': 1000,
-        'azimuth': 110, # Only used for visualization
+        'azimuth': 110,  # Only used for visualization
     },
     {
         'id': '60457274',
@@ -55,7 +56,7 @@ h36m_cameras_intrinsic_params = [
         'tangential_distortion': [-0.0005872055771760643, -0.0018133620033040643],
         'res_w': 1000,
         'res_h': 1002,
-        'azimuth': -110, # Only used for visualization
+        'azimuth': -110,  # Only used for visualization
     },
 ]
 
@@ -209,7 +210,7 @@ h36m_cameras_extrinsic_params = {
 class Human36mDataset(MocapDataset):
     def __init__(self, path, remove_static_joints=True):
         super().__init__(fps=50, skeleton=h36m_skeleton)
-        
+
         self._cameras = copy.deepcopy(h36m_cameras_extrinsic_params)
         for cameras in self._cameras.values():
             for i, cam in enumerate(cameras):
@@ -217,22 +218,23 @@ class Human36mDataset(MocapDataset):
                 for k, v in cam.items():
                     if k not in ['id', 'res_w', 'res_h']:
                         cam[k] = np.array(v, dtype='float32')
-                
+
                 # Normalize camera frame
-                cam['center'] = normalize_screen_coordinates(cam['center'], w=cam['res_w'], h=cam['res_h']).astype('float32')
-                cam['focal_length'] = cam['focal_length']/cam['res_w']*2
+                cam['center'] = normalize_screen_coordinates(cam['center'], w=cam['res_w'], h=cam['res_h']).astype(
+                    'float32')
+                cam['focal_length'] = cam['focal_length'] / cam['res_w'] * 2
                 if 'translation' in cam:
-                    cam['translation'] = cam['translation']/1000 # mm to meters
-                
+                    cam['translation'] = cam['translation'] / 1000  # mm to meters
+
                 # Add intrinsic parameters vector
                 cam['intrinsic'] = np.concatenate((cam['focal_length'],
                                                    cam['center'],
                                                    cam['radial_distortion'],
                                                    cam['tangential_distortion']))
-        
+
         # Load serialized dataset
         data = np.load(path, allow_pickle=True)['positions_3d'].item()
-        
+
         self._data = {}
         for subject, actions in data.items():
             self._data[subject] = {}
@@ -241,15 +243,14 @@ class Human36mDataset(MocapDataset):
                     'positions': positions,
                     'cameras': self._cameras[subject],
                 }
-                
+
         if remove_static_joints:
             # Bring the skeleton to 17 joints instead of the original 32
             self.remove_joints([4, 5, 9, 10, 11, 16, 20, 21, 22, 23, 24, 28, 29, 30, 31])
-            
+
             # Rewire shoulders to the correct parents
             self._skeleton._parents[11] = 8
             self._skeleton._parents[14] = 8
-            
+
     def supports_semi_supervised(self):
         return True
-   

backend/video_pose/common/humaneva_dataset.py

@@ -5,34 +5,35 @@
 # LICENSE file in the root directory of this source tree.
 #
 
-import numpy as np
 import copy
-from video_pose.common.skeleton import Skeleton
-from video_pose.common.mocap_dataset import MocapDataset
+import numpy as np
+
 from video_pose.common.camera import normalize_screen_coordinates, image_coordinates
-       
+from video_pose.common.mocap_dataset import MocapDataset
+from video_pose.common.skeleton import Skeleton
+
 humaneva_skeleton = Skeleton(parents=[-1, 0, 1, 2, 3, 1, 5, 6, 0, 8, 9, 0, 11, 12, 1],
-       joints_left=[2, 3, 4, 8, 9, 10],
-       joints_right=[5, 6, 7, 11, 12, 13])
+                             joints_left=[2, 3, 4, 8, 9, 10],
+                             joints_right=[5, 6, 7, 11, 12, 13])
 
 humaneva_cameras_intrinsic_params = [
     {
         'id': 'C1',
         'res_w': 640,
         'res_h': 480,
-        'azimuth': 0, # Only used for visualization
+        'azimuth': 0,  # Only used for visualization
     },
     {
         'id': 'C2',
         'res_w': 640,
         'res_h': 480,
-        'azimuth': -90, # Only used for visualization
+        'azimuth': -90,  # Only used for visualization
     },
     {
         'id': 'C3',
         'res_w': 640,
         'res_h': 480,
-        'azimuth': 90, # Only used for visualization
+        'azimuth': 90,  # Only used for visualization
     },
 ]
 
@@ -40,11 +41,11 @@ humaneva_cameras_extrinsic_params = {
     'S1': [
         {
             'orientation': [0.424207, -0.4983646, -0.5802981, 0.4847012],
-            'translation': [4062.227,  663.2477, 1528.397],
+            'translation': [4062.227, 663.2477, 1528.397],
         },
         {
             'orientation': [0.6503354, -0.7481602, -0.0919284, 0.0941766],
-            'translation': [844.8131, -3805.2092,  1504.9929],
+            'translation': [844.8131, -3805.2092, 1504.9929],
         },
         {
             'orientation': [0.0664734, -0.0690535, 0.7416416, -0.6639132],
@@ -53,30 +54,30 @@ humaneva_cameras_extrinsic_params = {
     ],
     'S2': [
         {
-            'orientation': [ 0.4214752, -0.4961493, -0.5838273, 0.4851187 ],
-            'translation': [ 4112.9121,   626.4929,  1545.2988], 
+            'orientation': [0.4214752, -0.4961493, -0.5838273, 0.4851187],
+            'translation': [4112.9121, 626.4929, 1545.2988],
         },
         {
-            'orientation': [ 0.6501393, -0.7476588, -0.0954617, 0.0959808 ],
-            'translation': [  923.5740, -3877.9243,  1504.5518], 
+            'orientation': [0.6501393, -0.7476588, -0.0954617, 0.0959808],
+            'translation': [923.5740, -3877.9243, 1504.5518],
         },
         {
-            'orientation': [ 0.0699353, -0.0712403, 0.7421637, -0.662742 ],
-            'translation': [ -781.4915,  3838.8853,  1444.9929], 
+            'orientation': [0.0699353, -0.0712403, 0.7421637, -0.662742],
+            'translation': [-781.4915, 3838.8853, 1444.9929],
         },
     ],
     'S3': [
         {
-            'orientation': [ 0.424207, -0.4983646, -0.5802981, 0.4847012 ],
-            'translation': [ 4062.2271,   663.2477,  1528.3970], 
+            'orientation': [0.424207, -0.4983646, -0.5802981, 0.4847012],
+            'translation': [4062.2271, 663.2477, 1528.3970],
         },
         {
-            'orientation': [ 0.6503354, -0.7481602, -0.0919284, 0.0941766 ],
-            'translation': [  844.8131, -3805.2092,  1504.9929], 
+            'orientation': [0.6503354, -0.7481602, -0.0919284, 0.0941766],
+            'translation': [844.8131, -3805.2092, 1504.9929],
         },
         {
-            'orientation': [ 0.0664734, -0.0690535, 0.7416416, -0.6639132 ],
-            'translation': [ -797.6738,  3916.3174,  1433.6602], 
+            'orientation': [0.0664734, -0.0690535, 0.7416416, -0.6639132],
+            'translation': [-797.6738, 3916.3174, 1433.6602],
         },
     ],
     'S4': [
@@ -84,13 +85,13 @@ humaneva_cameras_extrinsic_params = {
         {},
         {},
     ],
-    
+
 }
 
 class HumanEvaDataset(MocapDataset):
     def __init__(self, path):
         super().__init__(fps=60, skeleton=humaneva_skeleton)
-        
+
         self._cameras = copy.deepcopy(humaneva_cameras_extrinsic_params)
         for cameras in self._cameras.values():
             for i, cam in enumerate(cameras):
@@ -99,17 +100,17 @@ class HumanEvaDataset(MocapDataset):
                     if k not in ['id', 'res_w', 'res_h']:
                         cam[k] = np.array(v, dtype='float32')
                 if 'translation' in cam:
-                    cam['translation'] = cam['translation']/1000 # mm to meters
-                
+                    cam['translation'] = cam['translation'] / 1000  # mm to meters
+
         for subject in list(self._cameras.keys()):
             data = self._cameras[subject]
             del self._cameras[subject]
             for prefix in ['Train/', 'Validate/', 'Unlabeled/Train/', 'Unlabeled/Validate/', 'Unlabeled/']:
                 self._cameras[prefix + subject] = data
-        
+
         # Load serialized dataset
         data = np.load(path, allow_pickle=True)['positions_3d'].item()
-        
+
         self._data = {}
         for subject, actions in data.items():
             self._data[subject] = {}
@@ -118,4 +119,3 @@ class HumanEvaDataset(MocapDataset):
                     'positions': positions,
                     'cameras': self._cameras[subject],
                 }
-   

backend/video_pose/common/loss.py

@@ -5,8 +5,8 @@
 # LICENSE file in the root directory of this source tree.
 #
 
-import torch
 import numpy as np
+import torch
 
 def mpjpe(predicted, target):
     """
@@ -14,15 +14,15 @@ def mpjpe(predicted, target):
     often referred to as "Protocol #1" in many papers.
     """
     assert predicted.shape == target.shape
-    return torch.mean(torch.norm(predicted - target, dim=len(target.shape)-1))
-    
+    return torch.mean(torch.norm(predicted - target, dim=len(target.shape) - 1))
+
 def weighted_mpjpe(predicted, target, w):
     """
     Weighted mean per-joint position error (i.e. mean Euclidean distance)
     """
     assert predicted.shape == target.shape
     assert w.shape[0] == predicted.shape[0]
-    return torch.mean(w * torch.norm(predicted - target, dim=len(target.shape)-1))
+    return torch.mean(w * torch.norm(predicted - target, dim=len(target.shape) - 1))
 
 def p_mpjpe(predicted, target):
     """
@@ -30,16 +30,16 @@ def p_mpjpe(predicted, target):
     often referred to as "Protocol #2" in many papers.
     """
     assert predicted.shape == target.shape
-    
+
     muX = np.mean(target, axis=1, keepdims=True)
     muY = np.mean(predicted, axis=1, keepdims=True)
-    
+
     X0 = target - muX
     Y0 = predicted - muY
 
-    normX = np.sqrt(np.sum(X0**2, axis=(1, 2), keepdims=True))
-    normY = np.sqrt(np.sum(Y0**2, axis=(1, 2), keepdims=True))
-    
+    normX = np.sqrt(np.sum(X0 ** 2, axis=(1, 2), keepdims=True))
+    normY = np.sqrt(np.sum(Y0 ** 2, axis=(1, 2), keepdims=True))
+
     X0 /= normX
     Y0 /= normY
 
@@ -52,28 +52,28 @@ def p_mpjpe(predicted, target):
     sign_detR = np.sign(np.expand_dims(np.linalg.det(R), axis=1))
     V[:, :, -1] *= sign_detR
     s[:, -1] *= sign_detR.flatten()
-    R = np.matmul(V, U.transpose(0, 2, 1)) # Rotation
+    R = np.matmul(V, U.transpose(0, 2, 1))  # Rotation
 
     tr = np.expand_dims(np.sum(s, axis=1, keepdims=True), axis=2)
 
-    a = tr * normX / normY # Scale
-    t = muX - a*np.matmul(muY, R) # Translation
-    
+    a = tr * normX / normY  # Scale
+    t = muX - a * np.matmul(muY, R)  # Translation
+
     # Perform rigid transformation on the input
-    predicted_aligned = a*np.matmul(predicted, R) + t
-    
+    predicted_aligned = a * np.matmul(predicted, R) + t
+
     # Return MPJPE
-    return np.mean(np.linalg.norm(predicted_aligned - target, axis=len(target.shape)-1))
-    
+    return np.mean(np.linalg.norm(predicted_aligned - target, axis=len(target.shape) - 1))
+
 def n_mpjpe(predicted, target):
     """
     Normalized MPJPE (scale only), adapted from:
     https://github.com/hrhodin/UnsupervisedGeometryAwareRepresentationLearning/blob/master/losses/poses.py
     """
     assert predicted.shape == target.shape
-    
-    norm_predicted = torch.mean(torch.sum(predicted**2, dim=3, keepdim=True), dim=2, keepdim=True)
-    norm_target = torch.mean(torch.sum(target*predicted, dim=3, keepdim=True), dim=2, keepdim=True)
+
+    norm_predicted = torch.mean(torch.sum(predicted ** 2, dim=3, keepdim=True), dim=2, keepdim=True)
+    norm_target = torch.mean(torch.sum(target * predicted, dim=3, keepdim=True), dim=2, keepdim=True)
     scale = norm_target / norm_predicted
     return mpjpe(scale * predicted, target)
 
@@ -82,8 +82,8 @@ def mean_velocity_error(predicted, target):
     Mean per-joint velocity error (i.e. mean Euclidean distance of the 1st derivative)
     """
     assert predicted.shape == target.shape
-    
+
     velocity_predicted = np.diff(predicted, axis=0)
     velocity_target = np.diff(target, axis=0)
-    
-    return np.mean(np.linalg.norm(velocity_predicted - velocity_target, axis=len(target.shape)-1))
+
+    return np.mean(np.linalg.norm(velocity_predicted - velocity_target, axis=len(target.shape) - 1))

backend/video_pose/common/mocap_dataset.py

@@ -6,15 +6,16 @@
 #
 
 import numpy as np
+
 from video_pose.common.skeleton import Skeleton
 
 class MocapDataset:
     def __init__(self, fps, skeleton):
         self._skeleton = skeleton
         self._fps = fps
-        self._data = None # Must be filled by subclass
-        self._cameras = None # Must be filled by subclass
-    
+        self._data = None  # Must be filled by subclass
+        self._cameras = None  # Must be filled by subclass
+
     def remove_joints(self, joints_to_remove):
         kept_joints = self._skeleton.remove_joints(joints_to_remove)
         for subject in self._data.keys():
@@ -22,23 +23,22 @@ class MocapDataset:
                 s = self._data[subject][action]
                 if 'positions' in s:
                     s['positions'] = s['positions'][:, kept_joints]
-                
-        
+
     def __getitem__(self, key):
         return self._data[key]
-        
+
     def subjects(self):
         return self._data.keys()
-    
+
     def fps(self):
         return self._fps
-    
+
     def skeleton(self):
         return self._skeleton
-        
+
     def cameras(self):
         return self._cameras
-    
+
     def supports_semi_supervised(self):
         # This method can be overridden
-        return False
+        return False

backend/video_pose/common/model.py

@@ -11,33 +11,32 @@ class TemporalModelBase(nn.Module):
     """
     Do not instantiate this class.
     """
-    
+
     def __init__(self, num_joints_in, in_features, num_joints_out,
                  filter_widths, causal, dropout, channels):
         super().__init__()
-        
+
         # Validate input
         for fw in filter_widths:
             assert fw % 2 != 0, 'Only odd filter widths are supported'
-        
+
         self.num_joints_in = num_joints_in
         self.in_features = in_features
         self.num_joints_out = num_joints_out
         self.filter_widths = filter_widths
-        
+
         self.drop = nn.Dropout(dropout)
         self.relu = nn.ReLU(inplace=True)
-        
-        self.pad = [ filter_widths[0] // 2 ]
+
+        self.pad = [filter_widths[0] // 2]
         self.expand_bn = nn.BatchNorm1d(channels, momentum=0.1)
-        self.shrink = nn.Conv1d(channels, num_joints_out*3, 1)
-        
+        self.shrink = nn.Conv1d(channels, num_joints_out * 3, 1)
 
     def set_bn_momentum(self, momentum):
         self.expand_bn.momentum = momentum
         for bn in self.layers_bn:
             bn.momentum = momentum
-            
+
     def receptive_field(self):
         """
         Return the total receptive field of this model as # of frames.
@@ -45,8 +44,8 @@ class TemporalModelBase(nn.Module):
         frames = 0
         for f in self.pad:
             frames += f
-        return 1 + 2*frames
-    
+        return 1 + 2 * frames
+
     def total_causal_shift(self):
         """
         Return the asymmetric offset for sequence padding.
@@ -59,29 +58,29 @@ class TemporalModelBase(nn.Module):
             frames += self.causal_shift[i] * next_dilation
             next_dilation *= self.filter_widths[i]
         return frames
-        
+
     def forward(self, x):
         assert len(x.shape) == 4
         assert x.shape[-2] == self.num_joints_in
         assert x.shape[-1] == self.in_features
-        
+
         sz = x.shape[:3]
         x = x.view(x.shape[0], x.shape[1], -1)
         x = x.permute(0, 2, 1)
-        
+
         x = self._forward_blocks(x)
-        
+
         x = x.permute(0, 2, 1)
         x = x.view(sz[0], -1, self.num_joints_out, 3)
-        
-        return x    
+
+        return x
 
 class TemporalModel(TemporalModelBase):
     """
     Reference 3D pose estimation model with temporal convolutions.
     This implementation can be used for all use-cases.
     """
-    
+
     def __init__(self, num_joints_in, in_features, num_joints_out,
                  filter_widths, causal=False, dropout=0.25, channels=1024, dense=False):
         """
@@ -98,45 +97,45 @@ class TemporalModel(TemporalModelBase):
         dense -- use regular dense convolutions instead of dilated convolutions (ablation experiment)
         """
         super().__init__(num_joints_in, in_features, num_joints_out, filter_widths, causal, dropout, channels)
-        
-        self.expand_conv = nn.Conv1d(num_joints_in*in_features, channels, filter_widths[0], bias=False)
-        
+
+        self.expand_conv = nn.Conv1d(num_joints_in * in_features, channels, filter_widths[0], bias=False)
+
         layers_conv = []
         layers_bn = []
-        
-        self.causal_shift = [ (filter_widths[0]) // 2 if causal else 0 ]
+
+        self.causal_shift = [(filter_widths[0]) // 2 if causal else 0]
         next_dilation = filter_widths[0]
         for i in range(1, len(filter_widths)):
-            self.pad.append((filter_widths[i] - 1)*next_dilation // 2)
-            self.causal_shift.append((filter_widths[i]//2 * next_dilation) if causal else 0)
-            
+            self.pad.append((filter_widths[i] - 1) * next_dilation // 2)
+            self.causal_shift.append((filter_widths[i] // 2 * next_dilation) if causal else 0)
+
             layers_conv.append(nn.Conv1d(channels, channels,
-                                         filter_widths[i] if not dense else (2*self.pad[-1] + 1),
+                                         filter_widths[i] if not dense else (2 * self.pad[-1] + 1),
                                          dilation=next_dilation if not dense else 1,
                                          bias=False))
             layers_bn.append(nn.BatchNorm1d(channels, momentum=0.1))
             layers_conv.append(nn.Conv1d(channels, channels, 1, dilation=1, bias=False))
             layers_bn.append(nn.BatchNorm1d(channels, momentum=0.1))
-            
+
             next_dilation *= filter_widths[i]
-            
+
         self.layers_conv = nn.ModuleList(layers_conv)
         self.layers_bn = nn.ModuleList(layers_bn)
-        
+
     def _forward_blocks(self, x):
         x = self.drop(self.relu(self.expand_bn(self.expand_conv(x))))
-        
+
         for i in range(len(self.pad) - 1):
-            pad = self.pad[i+1]
-            shift = self.causal_shift[i+1]
-            res = x[:, :, pad + shift : x.shape[2] - pad + shift]
-            
-            x = self.drop(self.relu(self.layers_bn[2*i](self.layers_conv[2*i](x))))
-            x = res + self.drop(self.relu(self.layers_bn[2*i + 1](self.layers_conv[2*i + 1](x))))
-        
+            pad = self.pad[i + 1]
+            shift = self.causal_shift[i + 1]
+            res = x[:, :, pad + shift: x.shape[2] - pad + shift]
+
+            x = self.drop(self.relu(self.layers_bn[2 * i](self.layers_conv[2 * i](x))))
+            x = res + self.drop(self.relu(self.layers_bn[2 * i + 1](self.layers_conv[2 * i + 1](x))))
+
         x = self.shrink(x)
         return x
-    
+
 class TemporalModelOptimized1f(TemporalModelBase):
     """
     3D pose estimation model optimized for single-frame batching, i.e.
@@ -147,7 +146,7 @@ class TemporalModelOptimized1f(TemporalModelBase):
     to avoid generating unused intermediate results. The weights are interchangeable
     with the reference implementation.
     """
-    
+
     def __init__(self, num_joints_in, in_features, num_joints_out,
                  filter_widths, causal=False, dropout=0.25, channels=1024):
         """
@@ -163,35 +162,36 @@ class TemporalModelOptimized1f(TemporalModelBase):
         channels -- number of convolution channels
         """
         super().__init__(num_joints_in, in_features, num_joints_out, filter_widths, causal, dropout, channels)
-        
-        self.expand_conv = nn.Conv1d(num_joints_in*in_features, channels, filter_widths[0], stride=filter_widths[0], bias=False)
-        
+
+        self.expand_conv = nn.Conv1d(num_joints_in * in_features, channels, filter_widths[0], stride=filter_widths[0],
+                                     bias=False)
+
         layers_conv = []
         layers_bn = []
-        
-        self.causal_shift = [ (filter_widths[0] // 2) if causal else 0 ]
+
+        self.causal_shift = [(filter_widths[0] // 2) if causal else 0]
         next_dilation = filter_widths[0]
         for i in range(1, len(filter_widths)):
-            self.pad.append((filter_widths[i] - 1)*next_dilation // 2)
-            self.causal_shift.append((filter_widths[i]//2) if causal else 0)
-            
+            self.pad.append((filter_widths[i] - 1) * next_dilation // 2)
+            self.causal_shift.append((filter_widths[i] // 2) if causal else 0)
+
             layers_conv.append(nn.Conv1d(channels, channels, filter_widths[i], stride=filter_widths[i], bias=False))
             layers_bn.append(nn.BatchNorm1d(channels, momentum=0.1))
             layers_conv.append(nn.Conv1d(channels, channels, 1, dilation=1, bias=False))
             layers_bn.append(nn.BatchNorm1d(channels, momentum=0.1))
             next_dilation *= filter_widths[i]
-            
+
         self.layers_conv = nn.ModuleList(layers_conv)
         self.layers_bn = nn.ModuleList(layers_bn)
-        
+
     def _forward_blocks(self, x):
         x = self.drop(self.relu(self.expand_bn(self.expand_conv(x))))
-        
+
         for i in range(len(self.pad) - 1):
-            res = x[:, :, self.causal_shift[i+1] + self.filter_widths[i+1]//2 :: self.filter_widths[i+1]]
-            
-            x = self.drop(self.relu(self.layers_bn[2*i](self.layers_conv[2*i](x))))
-            x = res + self.drop(self.relu(self.layers_bn[2*i + 1](self.layers_conv[2*i + 1](x))))
-        
+            res = x[:, :, self.causal_shift[i + 1] + self.filter_widths[i + 1] // 2:: self.filter_widths[i + 1]]
+
+            x = self.drop(self.relu(self.layers_bn[2 * i](self.layers_conv[2 * i](x))))
+            x = res + self.drop(self.relu(self.layers_bn[2 * i + 1](self.layers_conv[2 * i + 1](x))))
+
         x = self.shrink(x)
-        return x
+        return x

backend/video_pose/common/quaternion.py

@@ -19,11 +19,10 @@ def qrot(q, v):
     assert q.shape[:-1] == v.shape[:-1]
 
     qvec = q[..., 1:]
-    uv = torch.cross(qvec, v, dim=len(q.shape)-1)
-    uuv = torch.cross(qvec, uv, dim=len(q.shape)-1)
+    uv = torch.cross(qvec, v, dim=len(q.shape) - 1)
+    uuv = torch.cross(qvec, uv, dim=len(q.shape) - 1)
     return (v + 2 * (q[..., :1] * uv + uuv))
-    
-    
+
 def qinverse(q, inplace=False):
     # We assume the quaternion to be normalized
     if inplace:
@@ -32,4 +31,4 @@ def qinverse(q, inplace=False):
     else:
         w = q[..., :1]
         xyz = q[..., 1:]
-        return torch.cat((w, -xyz), dim=len(q.shape)-1)
+        return torch.cat((w, -xyz), dim=len(q.shape) - 1)

backend/video_pose/common/skeleton.py

@@ -10,24 +10,24 @@ import numpy as np
 class Skeleton:
     def __init__(self, parents, joints_left, joints_right):
         assert len(joints_left) == len(joints_right)
-        
+
         self._parents = np.array(parents)
         self._joints_left = joints_left
         self._joints_right = joints_right
         self._compute_metadata()
-    
+
     def num_joints(self):
         return len(self._parents)
-    
+
     def parents(self):
         return self._parents
-    
+
     def has_children(self):
         return self._has_children
-    
+
     def children(self):
         return self._children
-    
+
     def remove_joints(self, joints_to_remove):
         """
         Remove the joints specified in 'joints_to_remove'.
@@ -40,7 +40,7 @@ class Skeleton:
         for i in range(len(self._parents)):
             while self._parents[i] in joints_to_remove:
                 self._parents[i] = self._parents[self._parents[i]]
-                
+
         index_offsets = np.zeros(len(self._parents), dtype=int)
         new_parents = []
         for i, parent in enumerate(self._parents):
@@ -49,8 +49,7 @@ class Skeleton:
             else:
                 index_offsets[i:] += 1
         self._parents = np.array(new_parents)
-        
-        
+
         if self._joints_left is not None:
             new_joints_left = []
             for joint in self._joints_left:
@@ -65,15 +64,15 @@ class Skeleton:
             self._joints_right = new_joints_right
 
         self._compute_metadata()
-        
+
         return valid_joints
-    
+
     def joints_left(self):
         return self._joints_left
-    
+
     def joints_right(self):
         return self._joints_right
-        
+
     def _compute_metadata(self):
         self._has_children = np.zeros(len(self._parents)).astype(bool)
         for i, parent in enumerate(self._parents):
@@ -85,4 +84,4 @@ class Skeleton:
             self._children.append([])
         for i, parent in enumerate(self._parents):
             if parent != -1:
-                self._children[parent].append(i)
+                self._children[parent].append(i)

backend/video_pose/common/utils.py

@@ -5,16 +5,16 @@
 # LICENSE file in the root directory of this source tree.
 #
 
-import torch
-import numpy as np
 import hashlib
+import numpy as np
+import torch
 
 def wrap(func, *args, unsqueeze=False):
     """
     Wrap a torch function so it can be called with NumPy arrays.
     Input and return types are seamlessly converted.
     """
-    
+
     # Convert input types where applicable
     args = list(args)
     for i, arg in enumerate(args):
@@ -22,9 +22,9 @@ def wrap(func, *args, unsqueeze=False):
             args[i] = torch.from_numpy(arg)
             if unsqueeze:
                 args[i] = args[i].unsqueeze(0)
-        
+
     result = func(*args)
-    
+
     # Convert output types where applicable
     if isinstance(result, tuple):
         result = list(result)
@@ -40,8 +40,8 @@ def wrap(func, *args, unsqueeze=False):
         return result.numpy()
     else:
         return result
-    
+
 def deterministic_random(min_value, max_value, data):
     digest = hashlib.sha256(data.encode()).digest()
     raw_value = int.from_bytes(digest[:4], byteorder='little', signed=False)
-    return int(raw_value / (2**32 - 1) * (max_value - min_value)) + min_value
+    return int(raw_value / (2 ** 32 - 1) * (max_value - min_value)) + min_value

backend/video_pose/common/visualization.py

@@ -6,6 +6,7 @@
 #
 
 import matplotlib
+
 matplotlib.use('Agg')
 
 import matplotlib.pyplot as plt
@@ -21,7 +22,7 @@ def get_resolution(filename):
         for line in pipe.stdout:
             w, h = line.decode().strip().split(',')
             return int(w), int(h)
-            
+
 def get_fps(filename):
     command = ['ffprobe', '-v', 'error', '-select_streams', 'v:0',
                '-show_entries', 'stream=r_frame_rate', '-of', 'csv=p=0', filename]
@@ -32,18 +33,18 @@ def get_fps(filename):
 
 def read_video(filename, skip=0, limit=-1):
     w, h = get_resolution(filename)
-    
+
     command = ['ffmpeg',
-            '-i', filename,
-            '-f', 'image2pipe',
-            '-pix_fmt', 'rgb24',
-            '-vsync', '0',
-            '-vcodec', 'rawvideo', '-']
-    
+               '-i', filename,
+               '-f', 'image2pipe',
+               '-pix_fmt', 'rgb24',
+               '-vsync', '0',
+               '-vcodec', 'rawvideo', '-']
+
     i = 0
-    with sp.Popen(command, stdout = sp.PIPE, bufsize=-1) as pipe:
+    with sp.Popen(command, stdout=sp.PIPE, bufsize=-1) as pipe:
         while True:
-            data = pipe.stdout.read(w*h*3)
+            data = pipe.stdout.read(w * h * 3)
             if not data:
                 break
             i += 1
@@ -51,12 +52,9 @@ def read_video(filename, skip=0, limit=-1):
                 continue
             if i > skip:
                 yield np.frombuffer(data, dtype='uint8').reshape((h, w, 3))
-            
-                
-                
-    
+
 def downsample_tensor(X, factor):
-    length = X.shape[0]//factor * factor
+    length = X.shape[0] // factor * factor
     return np.mean(X[:length].reshape(-1, factor, *X.shape[1:]), axis=1)
 
 def render_animation(keypoints, keypoints_metadata, poses, skeleton, fps, bitrate, azim, output, viewport,
@@ -71,7 +69,7 @@ def render_animation(keypoints, keypoints_metadata, poses, skeleton, fps, bitrat
      -- 'filename.gif': render and export the animation a gif file (requires imagemagick).
     """
     plt.ioff()
-    fig = plt.figure(figsize=(size*(1 + len(poses)), size))
+    fig = plt.figure(figsize=(size * (1 + len(poses)), size))
     ax_in = fig.add_subplot(1, 1 + len(poses), 1)
     ax_in.get_xaxis().set_visible(False)
     ax_in.get_yaxis().set_visible(False)
@@ -83,11 +81,11 @@ def render_animation(keypoints, keypoints_metadata, poses, skeleton, fps, bitrat
     trajectories = []
     radius = 1.7
     for index, (title, data) in enumerate(poses.items()):
-        ax = fig.add_subplot(1, 1 + len(poses), index+2, projection='3d')
+        ax = fig.add_subplot(1, 1 + len(poses), index + 2, projection='3d')
         ax.view_init(elev=15., azim=azim)
-        ax.set_xlim3d([-radius/2, radius/2])
+        ax.set_xlim3d([-radius / 2, radius / 2])
         ax.set_zlim3d([0, radius])
-        ax.set_ylim3d([-radius/2, radius/2])
+        ax.set_ylim3d([-radius / 2, radius / 2])
         try:
             ax.set_aspect('equal')
         except NotImplementedError:
@@ -96,7 +94,7 @@ def render_animation(keypoints, keypoints_metadata, poses, skeleton, fps, bitrat
         ax.set_yticklabels([])
         ax.set_zticklabels([])
         ax.dist = 7.5
-        ax.set_title(title) #, pad=35
+        ax.set_title(title)  # , pad=35
         ax_3d.append(ax)
         lines_3d.append([])
         trajectories.append(data[:, 0, [0, 1]])
@@ -113,14 +111,14 @@ def render_animation(keypoints, keypoints_metadata, poses, skeleton, fps, bitrat
             all_frames.append(f)
         effective_length = min(keypoints.shape[0], len(all_frames))
         all_frames = all_frames[:effective_length]
-        
-        keypoints = keypoints[input_video_skip:] # todo remove
+
+        keypoints = keypoints[input_video_skip:]  # todo remove
         for idx in range(len(poses)):
             poses[idx] = poses[idx][input_video_skip:]
-        
+
         if fps is None:
             fps = get_fps(input_video_path)
-    
+
     if downsample > 1:
         keypoints = downsample_tensor(keypoints, downsample)
         all_frames = downsample_tensor(np.array(all_frames), downsample).astype('uint8')
@@ -133,19 +131,20 @@ def render_animation(keypoints, keypoints_metadata, poses, skeleton, fps, bitrat
     image = None
     lines = []
     points = None
-    
+
     if limit < 1:
         limit = len(all_frames)
     else:
         limit = min(limit, len(all_frames))
 
     parents = skeleton.parents()
+
     def update_video(i):
         nonlocal initialized, image, lines, points
 
         for n, ax in enumerate(ax_3d):
-            ax.set_xlim3d([-radius/2 + trajectories[n][i, 0], radius/2 + trajectories[n][i, 0]])
-            ax.set_ylim3d([-radius/2 + trajectories[n][i, 1], radius/2 + trajectories[n][i, 1]])
+            ax.set_xlim3d([-radius / 2 + trajectories[n][i, 0], radius / 2 + trajectories[n][i, 0]])
+            ax.set_ylim3d([-radius / 2 + trajectories[n][i, 1], radius / 2 + trajectories[n][i, 1]])
 
         # Update 2D poses
         joints_right_2d = keypoints_metadata['keypoints_symmetry'][1]
@@ -153,11 +152,11 @@ def render_animation(keypoints, keypoints_metadata, poses, skeleton, fps, bitrat
         colors_2d[joints_right_2d] = 'red'
         if not initialized:
             image = ax_in.imshow(all_frames[i], aspect='equal')
-            
+
             for j, j_parent in enumerate(parents):
                 if j_parent == -1:
                     continue
-                    
+
                 if len(parents) == keypoints.shape[1] and keypoints_metadata['layout_name'] != 'coco':
                     # Draw skeleton only if keypoints match (otherwise we don't have the parents definition)
                     lines.append(ax_in.plot([keypoints[i, j, 0], keypoints[i, j_parent, 0]],
@@ -179,25 +178,24 @@ def render_animation(keypoints, keypoints_metadata, poses, skeleton, fps, bitrat
             for j, j_parent in enumerate(parents):
                 if j_parent == -1:
                     continue
-                
+
                 if len(parents) == keypoints.shape[1] and keypoints_metadata['layout_name'] != 'coco':
-                    lines[j-1][0].set_data([keypoints[i, j, 0], keypoints[i, j_parent, 0]],
-                                           [keypoints[i, j, 1], keypoints[i, j_parent, 1]])
+                    lines[j - 1][0].set_data([keypoints[i, j, 0], keypoints[i, j_parent, 0]],
+                                             [keypoints[i, j, 1], keypoints[i, j_parent, 1]])
 
                 for n, ax in enumerate(ax_3d):
                     pos = poses[n][i]
-                    lines_3d[n][j-1][0].set_xdata(np.array([pos[j, 0], pos[j_parent, 0]]))
-                    lines_3d[n][j-1][0].set_ydata(np.array([pos[j, 1], pos[j_parent, 1]]))
-                    lines_3d[n][j-1][0].set_3d_properties(np.array([pos[j, 2], pos[j_parent, 2]]), zdir='z')
+                    lines_3d[n][j - 1][0].set_xdata(np.array([pos[j, 0], pos[j_parent, 0]]))
+                    lines_3d[n][j - 1][0].set_ydata(np.array([pos[j, 1], pos[j_parent, 1]]))
+                    lines_3d[n][j - 1][0].set_3d_properties(np.array([pos[j, 2], pos[j_parent, 2]]), zdir='z')
 
             points.set_offsets(keypoints[i])
-        
+
         print('{}/{}      '.format(i, limit), end='\r')
-        
 
     fig.tight_layout()
-    
-    anim = FuncAnimation(fig, update_video, frames=np.arange(0, limit), interval=1000/fps, repeat=False)
+
+    anim = FuncAnimation(fig, update_video, frames=np.arange(0, limit), interval=1000 / fps, repeat=False)
     if output.endswith('.mp4'):
         Writer = writers['ffmpeg']
         writer = Writer(fps=fps, metadata={}, bitrate=bitrate)
@@ -206,4 +204,4 @@ def render_animation(keypoints, keypoints_metadata, poses, skeleton, fps, bitrat
         anim.save(output, dpi=80, writer='imagemagick')
     else:
         raise ValueError('Unsupported output format (only .mp4 and .gif are supported)')
-    plt.close()
+    plt.close()

backend/video_pose/data/data_utils.py

@@ -76,22 +76,20 @@ def import_detectron_poses(path):
         keypoints = kp[i][1][best_match].T.copy()
         results.append(keypoints)
     results = np.array(results)
-    return results[:, :, 4:6] # Soft-argmax
-    #return results[:, :, [0, 1, 3]] # Argmax + score
-    
-    
+    return results[:, :, 4:6]  # Soft-argmax
+    # return results[:, :, [0, 1, 3]] # Argmax + score
+
 def import_cpn_poses(path):
     data = np.load(path)
     kp = data['keypoints']
     return kp[:, :, :2]
-    
-    
+
 def import_sh_poses(path):
     import h5py
     with h5py.File(path) as hf:
         positions = hf['poses'].value
     return positions.astype('float32')
-    
+
 def suggest_pose_importer(name):
     if 'detectron' in name:
         return import_detectron_poses

backend/video_pose/data/prepare_data_2d_custom.py

@@ -5,15 +5,14 @@
 # LICENSE file in the root directory of this source tree.
 #
 
+import argparse
 import numpy as np
-from glob import glob
 import os
 import sys
+from glob import glob
 
-import argparse
 from video_pose.data.data_utils import suggest_metadata
 
-
 # output_prefix_2d = 'data_2d_custom_'
 
 def decode(filename):
@@ -61,7 +60,6 @@ def decode(filename):
         'keypoints': kp,
     }], metadata
 
-
 def custom(video_analyzed_path):
     metadata = suggest_metadata('coco')
     metadata['video_metadata'] = {}
@@ -79,7 +77,6 @@ def custom(video_analyzed_path):
     np.savez_compressed(output_path, positions_2d=output, metadata=metadata)
     print('Done.')
 
-
 if __name__ == '__main__':
     pass
     # if os.path.basename(os.getcwd()) != 'data':

backend/video_pose/data/prepare_data_2d_h36m_generic.py

@@ -6,16 +6,17 @@
 #
 
 import argparse
-import os
-import zipfile
-import numpy as np
 import h5py
+import numpy as np
+import os
 import re
+import sys
+import zipfile
 from glob import glob
 from shutil import rmtree
+
 from data_utils import suggest_metadata, suggest_pose_importer
 
-import sys
 sys.path.append('../')
 from common.utils import wrap
 from itertools import groupby
@@ -32,18 +33,19 @@ if __name__ == '__main__':
     if os.path.basename(os.getcwd()) != 'data':
         print('This script must be launched from the "data" directory')
         exit(0)
-        
+
     parser = argparse.ArgumentParser(description='Human3.6M dataset converter')
-    
+
     parser.add_argument('-i', '--input', default='', type=str, metavar='PATH', help='input path to 2D detections')
-    parser.add_argument('-o', '--output', default='', type=str, metavar='PATH', help='output suffix for 2D detections (e.g. detectron_pt_coco)')
-    
+    parser.add_argument('-o', '--output', default='', type=str, metavar='PATH',
+                        help='output suffix for 2D detections (e.g. detectron_pt_coco)')
+
     args = parser.parse_args()
-        
+
     if not args.input:
         print('Please specify the input directory')
         exit(0)
-        
+
     if not args.output:
         print('Please specify an output suffix (e.g. detectron_pt_coco)')
         exit(0)
@@ -62,22 +64,22 @@ if __name__ == '__main__':
 
         if '_ALL' in fname:
             continue
-        
+
         m = re.search('(.*)\\.([0-9]+)\\.mp4\\.npz', fname)
         action = m.group(1)
         camera = m.group(2)
         camera_idx = cam_map[camera]
-        
+
         if subject == 'S11' and action == 'Directions':
-            continue # Discard corrupted video
-        
+            continue  # Discard corrupted video
+
         # Use consistent naming convention
         canonical_name = action.replace('TakingPhoto', 'Photo') \
-                               .replace('WalkingDog', 'WalkDog')
+            .replace('WalkingDog', 'WalkDog')
 
         keypoints = import_func(f)
         assert keypoints.shape[1] == metadata['num_joints']
-        
+
         if subject not in output:
             output[subject] = {}
         if canonical_name not in output[subject]:
@@ -86,4 +88,4 @@ if __name__ == '__main__':
 
     print('Saving...')
     np.savez_compressed(output_prefix_2d + args.output, positions_2d=output, metadata=metadata)
-    print('Done.')
+    print('Done.')

backend/video_pose/data/prepare_data_2d_h36m_sh.py

@@ -6,15 +6,15 @@
 #
 
 import argparse
+import h5py
+import numpy as np
 import os
-import zipfile
+import sys
 import tarfile
-import numpy as np
-import h5py
+import zipfile
 from glob import glob
 from shutil import rmtree
 
-import sys
 sys.path.append('../')
 from common.h36m_dataset import Human36mDataset
 from common.camera import world_to_camera, project_to_2d, image_coordinates
@@ -43,70 +43,70 @@ def process_subject(subject, file_list, output):
         assert len(file_list) == 119, "Expected 119 files for subject " + subject + ", got " + str(len(file_list))
     else:
         assert len(file_list) == 120, "Expected 120 files for subject " + subject + ", got " + str(len(file_list))
-        
+
     for f in file_list:
         action, cam = os.path.splitext(os.path.basename(f))[0].replace('_', ' ').split('.')
-        
+
         if subject == 'S11' and action == 'Directions':
-            continue # Discard corrupted video
-        
+            continue  # Discard corrupted video
+
         if action not in output[subject]:
             output[subject][action] = [None, None, None, None]
-        
+
         with h5py.File(f) as hf:
             positions = hf['poses'].value
-            output[subject][action][cam_map[cam]] = positions.astype('float32')    
+            output[subject][action][cam_map[cam]] = positions.astype('float32')
 
 if __name__ == '__main__':
     if os.path.basename(os.getcwd()) != 'data':
         print('This script must be launched from the "data" directory')
         exit(0)
-        
+
     parser = argparse.ArgumentParser(description='Human3.6M dataset downloader/converter')
-    
+
     parser.add_argument('-pt', '--pretrained', default='', type=str, metavar='PATH', help='convert pretrained dataset')
     parser.add_argument('-ft', '--fine-tuned', default='', type=str, metavar='PATH', help='convert fine-tuned dataset')
-    
+
     args = parser.parse_args()
-        
+
     if args.pretrained:
         print('Converting pretrained dataset from', args.pretrained)
         print('Extracting...')
         with zipfile.ZipFile(args.pretrained, 'r') as archive:
             archive.extractall('sh_pt')
-        
+
         print('Converting...')
         output = {}
         for subject in subjects:
             output[subject] = {}
             file_list = glob('sh_pt/h36m/' + subject + '/StackedHourglass/*.h5')
             process_subject(subject, file_list, output)
-        
+
         print('Saving...')
         np.savez_compressed(output_filename_pt, positions_2d=output, metadata=metadata)
-        
+
         print('Cleaning up...')
         rmtree('sh_pt')
-        
+
         print('Done.')
-        
+
     if args.fine_tuned:
         print('Converting fine-tuned dataset from', args.fine_tuned)
         print('Extracting...')
         with tarfile.open(args.fine_tuned, 'r:gz') as archive:
             archive.extractall('sh_ft')
-        
+
         print('Converting...')
         output = {}
         for subject in subjects:
             output[subject] = {}
             file_list = glob('sh_ft/' + subject + '/StackedHourglassFineTuned240/*.h5')
             process_subject(subject, file_list, output)
-        
+
         print('Saving...')
         np.savez_compressed(output_filename_ft, positions_2d=output, metadata=metadata)
-        
+
         print('Cleaning up...')
         rmtree('sh_ft')
-        
+
         print('Done.')

backend/video_pose/data/prepare_data_h36m.py

@@ -6,14 +6,14 @@
 #
 
 import argparse
+import h5py
+import numpy as np
 import os
+import sys
 import zipfile
-import numpy as np
-import h5py
 from glob import glob
 from shutil import rmtree
 
-import sys
 sys.path.append('../')
 from common.h36m_dataset import Human36mDataset
 from common.camera import world_to_camera, project_to_2d, image_coordinates
@@ -27,35 +27,35 @@ if __name__ == '__main__':
     if os.path.basename(os.getcwd()) != 'data':
         print('This script must be launched from the "data" directory')
         exit(0)
-        
+
     parser = argparse.ArgumentParser(description='Human3.6M dataset downloader/converter')
-    
+
     # Convert dataset preprocessed by Martinez et al. in https://github.com/una-dinosauria/3d-pose-baseline
     parser.add_argument('--from-archive', default='', type=str, metavar='PATH', help='convert preprocessed dataset')
-    
+
     # Convert dataset from original source, using files converted to .mat (the Human3.6M dataset path must be specified manually)
     # This option requires MATLAB to convert files using the provided script
     parser.add_argument('--from-source', default='', type=str, metavar='PATH', help='convert original dataset')
-    
+
     # Convert dataset from original source, using original .cdf files (the Human3.6M dataset path must be specified manually)
     # This option does not require MATLAB, but the Python library cdflib must be installed
     parser.add_argument('--from-source-cdf', default='', type=str, metavar='PATH', help='convert original dataset')
-    
+
     args = parser.parse_args()
-    
+
     if args.from_archive and args.from_source:
         print('Please specify only one argument')
         exit(0)
-    
+
     if os.path.exists(output_filename + '.npz'):
         print('The dataset already exists at', output_filename + '.npz')
         exit(0)
-        
+
     if args.from_archive:
         print('Extracting Human3.6M dataset from', args.from_archive)
         with zipfile.ZipFile(args.from_archive, 'r') as archive:
             archive.extractall()
-        
+
         print('Converting...')
         output = {}
         for subject in subjects:
@@ -64,87 +64,87 @@ if __name__ == '__main__':
             assert len(file_list) == 30, "Expected 30 files for subject " + subject + ", got " + str(len(file_list))
             for f in file_list:
                 action = os.path.splitext(os.path.basename(f))[0]
-                
+
                 if subject == 'S11' and action == 'Directions':
-                    continue # Discard corrupted video
-                
+                    continue  # Discard corrupted video
+
                 with h5py.File(f) as hf:
                     positions = hf['3D_positions'].value.reshape(32, 3, -1).transpose(2, 0, 1)
-                    positions /= 1000 # Meters instead of millimeters
+                    positions /= 1000  # Meters instead of millimeters
                     output[subject][action] = positions.astype('float32')
-        
+
         print('Saving...')
         np.savez_compressed(output_filename, positions_3d=output)
-        
+
         print('Cleaning up...')
         rmtree('h36m')
-        
+
         print('Done.')
-                
+
     elif args.from_source:
         print('Converting original Human3.6M dataset from', args.from_source)
         output = {}
-        
+
         from scipy.io import loadmat
-        
+
         for subject in subjects:
             output[subject] = {}
             file_list = glob(args.from_source + '/' + subject + '/MyPoseFeatures/D3_Positions/*.cdf.mat')
             assert len(file_list) == 30, "Expected 30 files for subject " + subject + ", got " + str(len(file_list))
             for f in file_list:
                 action = os.path.splitext(os.path.splitext(os.path.basename(f))[0])[0]
-                
+
                 if subject == 'S11' and action == 'Directions':
-                    continue # Discard corrupted video
-                    
+                    continue  # Discard corrupted video
+
                 # Use consistent naming convention
                 canonical_name = action.replace('TakingPhoto', 'Photo') \
-                                       .replace('WalkingDog', 'WalkDog')
-                
+                    .replace('WalkingDog', 'WalkDog')
+
                 hf = loadmat(f)
                 positions = hf['data'][0, 0].reshape(-1, 32, 3)
-                positions /= 1000 # Meters instead of millimeters
+                positions /= 1000  # Meters instead of millimeters
                 output[subject][canonical_name] = positions.astype('float32')
-        
+
         print('Saving...')
         np.savez_compressed(output_filename, positions_3d=output)
-        
+
         print('Done.')
-        
+
     elif args.from_source_cdf:
         print('Converting original Human3.6M dataset from', args.from_source_cdf, '(CDF files)')
         output = {}
-        
+
         import cdflib
-        
+
         for subject in subjects:
             output[subject] = {}
             file_list = glob(args.from_source_cdf + '/' + subject + '/MyPoseFeatures/D3_Positions/*.cdf')
             assert len(file_list) == 30, "Expected 30 files for subject " + subject + ", got " + str(len(file_list))
             for f in file_list:
                 action = os.path.splitext(os.path.basename(f))[0]
-                
+
                 if subject == 'S11' and action == 'Directions':
-                    continue # Discard corrupted video
-                    
+                    continue  # Discard corrupted video
+
                 # Use consistent naming convention
                 canonical_name = action.replace('TakingPhoto', 'Photo') \
-                                       .replace('WalkingDog', 'WalkDog')
-                
+                    .replace('WalkingDog', 'WalkDog')
+
                 hf = cdflib.CDF(f)
                 positions = hf['Pose'].reshape(-1, 32, 3)
-                positions /= 1000 # Meters instead of millimeters
+                positions /= 1000  # Meters instead of millimeters
                 output[subject][canonical_name] = positions.astype('float32')
-        
+
         print('Saving...')
         np.savez_compressed(output_filename, positions_3d=output)
-        
+
         print('Done.')
-            
+
     else:
         print('Please specify the dataset source')
         exit(0)
-        
+
     # Create 2D pose file
     print('')
     print('Computing ground-truth 2D poses...')
@@ -154,7 +154,7 @@ if __name__ == '__main__':
         output_2d_poses[subject] = {}
         for action in dataset[subject].keys():
             anim = dataset[subject][action]
-            
+
             positions_2d = []
             for cam in anim['cameras']:
                 pos_3d = world_to_camera(anim['positions'], R=cam['orientation'], t=cam['translation'])
@@ -162,12 +162,12 @@ if __name__ == '__main__':
                 pos_2d_pixel_space = image_coordinates(pos_2d, w=cam['res_w'], h=cam['res_h'])
                 positions_2d.append(pos_2d_pixel_space.astype('float32'))
             output_2d_poses[subject][action] = positions_2d
-            
+
     print('Saving...')
     metadata = {
         'num_joints': dataset.skeleton().num_joints(),
         'keypoints_symmetry': [dataset.skeleton().joints_left(), dataset.skeleton().joints_right()]
     }
     np.savez_compressed(output_filename_2d, positions_2d=output_2d_poses, metadata=metadata)
-    
+
     print('Done.')

backend/video_pose/data/prepare_data_humaneva.py

@@ -6,16 +6,17 @@
 #
 
 import argparse
-import os
-import zipfile
-import numpy as np
 import h5py
+import numpy as np
+import os
 import re
+import sys
+import zipfile
 from glob import glob
 from shutil import rmtree
+
 from data_utils import suggest_metadata, suggest_pose_importer
 
-import sys
 sys.path.append('../')
 from common.utils import wrap
 from itertools import groupby
@@ -105,55 +106,57 @@ if __name__ == '__main__':
     if os.path.basename(os.getcwd()) != 'data':
         print('This script must be launched from the "data" directory')
         exit(0)
-        
+
     parser = argparse.ArgumentParser(description='HumanEva dataset converter')
-    
-    parser.add_argument('-p', '--path', default='', type=str, metavar='PATH', help='path to the processed HumanEva dataset')
+
+    parser.add_argument('-p', '--path', default='', type=str, metavar='PATH',
+                        help='path to the processed HumanEva dataset')
     parser.add_argument('--convert-3d', action='store_true', help='convert 3D mocap data')
-    parser.add_argument('--convert-2d', default='', type=str, metavar='PATH', help='convert user-supplied 2D detections')
-    parser.add_argument('-o', '--output', default='', type=str, metavar='PATH', help='output suffix for 2D detections (e.g. detectron_pt_coco)')
-    
+    parser.add_argument('--convert-2d', default='', type=str, metavar='PATH',
+                        help='convert user-supplied 2D detections')
+    parser.add_argument('-o', '--output', default='', type=str, metavar='PATH',
+                        help='output suffix for 2D detections (e.g. detectron_pt_coco)')
+
     args = parser.parse_args()
-        
+
     if not args.convert_2d and not args.convert_3d:
         print('Please specify one conversion mode')
         exit(0)
-        
- 
+
     if args.path:
         print('Parsing HumanEva dataset from', args.path)
         output = {}
         output_2d = {}
         frame_mapping = {}
-        
+
         from scipy.io import loadmat
-        
+
         num_joints = None
-        
+
         for subject in subjects:
             output[subject] = {}
             output_2d[subject] = {}
             split, subject_name = subject.split('/')
             if subject_name not in frame_mapping:
                 frame_mapping[subject_name] = {}
-            
+
             file_list = glob(args.path + '/' + subject + '/*.mat')
             for f in file_list:
                 action = os.path.splitext(os.path.basename(f))[0]
-                    
+
                 # Use consistent naming convention
                 canonical_name = action.replace('_', ' ')
-                
+
                 hf = loadmat(f)
                 positions = hf['poses_3d']
-                positions_2d = hf['poses_2d'].transpose(1, 0, 2, 3) # Ground-truth 2D poses
+                positions_2d = hf['poses_2d'].transpose(1, 0, 2, 3)  # Ground-truth 2D poses
                 assert positions.shape[0] == positions_2d.shape[0] and positions.shape[1] == positions_2d.shape[2]
                 assert num_joints is None or num_joints == positions.shape[1], "Joint number inconsistency among files"
                 num_joints = positions.shape[1]
-                
+
                 # Sanity check for the sequence length
                 assert positions.shape[0] == index[subject][canonical_name][1] - index[subject][canonical_name][0]
-                
+
                 # Split corrupted motion capture streams into contiguous chunks
                 # e.g. 012XX567X9 is split into "012", "567", and "9".
                 all_chunks = [list(v) for k, v in groupby(positions, lambda x: np.isfinite(x).all())]
@@ -166,53 +169,54 @@ if __name__ == '__main__':
                     name = canonical_name + ' chunk' + str(i)
                     if np.isfinite(chunk).all():
                         output[subject][name] = np.array(chunk, dtype='float32') / 1000
-                        output_2d[subject][name] = list(np.array(all_chunks_2d[i], dtype='float32').transpose(1, 0, 2, 3))
+                        output_2d[subject][name] = list(
+                            np.array(all_chunks_2d[i], dtype='float32').transpose(1, 0, 2, 3))
                     chunk_indices.append((current_index, next_index, np.isfinite(chunk).all(), split, name))
                     current_index = next_index
                 assert current_index == index[subject][canonical_name][1]
                 if canonical_name not in frame_mapping[subject_name]:
                     frame_mapping[subject_name][canonical_name] = []
                 frame_mapping[subject_name][canonical_name] += chunk_indices
-        
+
         metadata = suggest_metadata('humaneva' + str(num_joints))
         output_filename = 'data_3d_' + metadata['layout_name']
         output_prefix_2d = 'data_2d_' + metadata['layout_name'] + '_'
-        
+
         if args.convert_3d:
             print('Saving...')
             np.savez_compressed(output_filename, positions_3d=output)
             np.savez_compressed(output_prefix_2d + 'gt', positions_2d=output_2d, metadata=metadata)
             print('Done.')
-        
+
     else:
         print('Please specify the dataset source')
         exit(0)
-        
+
     if args.convert_2d:
         if not args.output:
             print('Please specify an output suffix (e.g. detectron_pt_coco)')
             exit(0)
-            
+
         import_func = suggest_pose_importer(args.output)
         metadata = suggest_metadata(args.output)
-            
+
         print('Parsing 2D detections from', args.convert_2d)
-        
+
         output = {}
         file_list = glob(args.convert_2d + '/S*/*.avi.npz')
         for f in file_list:
             path, fname = os.path.split(f)
             subject = os.path.basename(path)
             assert subject.startswith('S'), subject + ' does not look like a subject directory'
-            
+
             m = re.search('(.*) \\((.*)\\)', fname.replace('_', ' '))
             action = m.group(1)
             camera = m.group(2)
             camera_idx = cam_map[camera]
-            
+
             keypoints = import_func(f)
             assert keypoints.shape[1] == metadata['num_joints']
-            
+
             if action in sync_data[subject]:
                 sync_offset = sync_data[subject][action][camera_idx] - 1
             else:
@@ -226,21 +230,22 @@ if __name__ == '__main__':
                         canonical_subject = 'Unlabeled/' + canonical_subject
                     if canonical_subject not in output:
                         output[canonical_subject] = {}
-                    kps = keypoints[start_idx+sync_offset:end_idx+sync_offset]
-                    assert len(kps) == end_idx - start_idx, "Got len {}, expected {}".format(len(kps), end_idx - start_idx)
-                    
+                    kps = keypoints[start_idx + sync_offset:end_idx + sync_offset]
+                    assert len(kps) == end_idx - start_idx, "Got len {}, expected {}".format(len(kps),
+                                                                                             end_idx - start_idx)
+
                     if name not in output[canonical_subject]:
                         output[canonical_subject][name] = [None, None, None]
-                    
+
                     output[canonical_subject][name][camera_idx] = kps.astype('float32')
             else:
                 canonical_subject = 'Unlabeled/' + subject
                 if canonical_subject not in output:
                     output[canonical_subject] = {}
                 if action not in output[canonical_subject]:
-                        output[canonical_subject][action] = [None, None, None]
+                    output[canonical_subject][action] = [None, None, None]
                 output[canonical_subject][action][camera_idx] = keypoints.astype('float32')
-                
+
         print('Saving...')
         np.savez_compressed(output_prefix_2d + args.output, positions_2d=output, metadata=metadata)
-        print('Done.')
+        print('Done.')

backend/video_pose/inference/infer_video.py

@@ -9,9 +9,9 @@
 (e.g., .mp4) in a folder.
 """
 
-from infer_simple import *
-import subprocess as sp
 import numpy as np
+import subprocess as sp
+from infer_simple import *
 
 def get_resolution(filename):
     command = ['ffprobe', '-v', 'error', '-select_streams', 'v:0',
@@ -25,22 +25,20 @@ def read_video(filename):
     w, h = get_resolution(filename)
 
     command = ['ffmpeg',
-            '-i', filename,
-            '-f', 'image2pipe',
-            '-pix_fmt', 'bgr24',
-            '-vsync', '0',
-            '-vcodec', 'rawvideo', '-']
+               '-i', filename,
+               '-f', 'image2pipe',
+               '-pix_fmt', 'bgr24',
+               '-vsync', '0',
+               '-vcodec', 'rawvideo', '-']
 
     pipe = sp.Popen(command, stdout=sp.PIPE, bufsize=-1)
     while True:
-        data = pipe.stdout.read(w*h*3)
+        data = pipe.stdout.read(w * h * 3)
         if not data:
             break
         yield np.frombuffer(data, dtype='uint8').reshape((h, w, 3))
 
-
 def main(args):
-
     logger = logging.getLogger(__name__)
     merge_cfg_from_file(args.cfg)
     cfg.NUM_GPUS = 1
@@ -49,8 +47,6 @@ def main(args):
     model = infer_engine.initialize_model_from_cfg(args.weights)
     dummy_coco_dataset = dummy_datasets.get_coco_dataset()
 
-
-
     if os.path.isdir(args.im_or_folder):
         im_list = glob.iglob(args.im_or_folder + '/*.' + args.image_ext)
     else:
@@ -58,8 +54,8 @@ def main(args):
 
     for video_name in im_list:
         out_name = os.path.join(
-                args.output_dir, os.path.basename(video_name)
-            )
+            args.output_dir, os.path.basename(video_name)
+        )
         print('Processing {}'.format(video_name))
 
         boxes = []
@@ -83,15 +79,13 @@ def main(args):
             segments.append(cls_segms)
             keypoints.append(cls_keyps)
 
-        
         # Video resolution
         metadata = {
             'w': im.shape[1],
             'h': im.shape[0],
         }
-        
-        np.savez_compressed(out_name, boxes=boxes, segments=segments, keypoints=keypoints, metadata=metadata)
 
+        np.savez_compressed(out_name, boxes=boxes, segments=segments, keypoints=keypoints, metadata=metadata)
 
 if __name__ == '__main__':
     workspace.GlobalInit(['caffe2', '--caffe2_log_level=0'])

backend/video_pose/inference/infer_video_d2.py

@@ -9,19 +9,18 @@
 (e.g., .mp4) in a folder.
 """
 
+import argparse
 import detectron2
-from detectron2.utils.logger import setup_logger
-from detectron2.config import get_cfg
-from detectron2 import model_zoo
-from detectron2.engine import DefaultPredictor
-
-import subprocess as sp
+import glob
 import numpy as np
-import time
-import argparse
-import sys
 import os
-import glob
+import subprocess as sp
+import sys
+import time
+from detectron2 import model_zoo
+from detectron2.config import get_cfg
+from detectron2.engine import DefaultPredictor
+from detectron2.utils.logger import setup_logger
 
 # def parse_args():
 #     parser = argparse.ArgumentParser(description='End-to-end inference')
@@ -66,20 +65,19 @@ def read_video(filename):
     w, h = get_resolution(filename)
 
     command = ['ffmpeg',
-            '-i', filename,
-            '-f', 'image2pipe',
-            '-pix_fmt', 'bgr24',
-            '-vsync', '0',
-            '-vcodec', 'rawvideo', '-']
+               '-i', filename,
+               '-f', 'image2pipe',
+               '-pix_fmt', 'bgr24',
+               '-vsync', '0',
+               '-vcodec', 'rawvideo', '-']
 
     pipe = sp.Popen(command, stdout=sp.PIPE, bufsize=-1)
     while True:
-        data = pipe.stdout.read(w*h*3)
+        data = pipe.stdout.read(w * h * 3)
         if not data:
             break
         yield np.frombuffer(data, dtype='uint8').reshape((h, w, 3))
 
-
 def run_3d(file_name):
     # todo: ？？？
     cfg = get_cfg()
@@ -89,7 +87,7 @@ def run_3d(file_name):
     cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url(cfg_path)
     predictor = DefaultPredictor(cfg)
     input_folder = file_name
-    output_folder = file_name.replace(os.path.basename(file_name),"")
+    output_folder = file_name.replace(os.path.basename(file_name), "")
 
     if os.path.isdir(input_folder):
         im_list = glob.iglob(input_folder + '/*.mp4')
@@ -98,8 +96,8 @@ def run_3d(file_name):
 
     for video_name in im_list:
         out_name = os.path.join(
-                output_folder, os.path.basename(video_name)
-            )
+            output_folder, os.path.basename(video_name)
+        )
         print('Processing {}'.format(video_name))
 
         boxes = []
@@ -123,7 +121,7 @@ def run_3d(file_name):
                 kps = outputs.pred_keypoints.numpy()
                 kps_xy = kps[:, :, :2]
                 kps_prob = kps[:, :, 2:3]
-                kps_logit = np.zeros_like(kps_prob) # Dummy
+                kps_logit = np.zeros_like(kps_prob)  # Dummy
                 kps = np.concatenate((kps_xy, kps_logit, kps_prob), axis=2)
                 kps = kps.transpose(0, 2, 1)
             else:
@@ -138,7 +136,6 @@ def run_3d(file_name):
             segments.append(None)
             keypoints.append(cls_keyps)
 
-
         # Video resolution
         metadata = {
             'w': im.shape[1],
@@ -147,7 +144,6 @@ def run_3d(file_name):
 
         np.savez_compressed(out_name, boxes=boxes, segments=segments, keypoints=keypoints, metadata=metadata)
 
-
 if __name__ == '__main__':
     setup_logger()
     file_name = "original.mp4"

backend/video_pose/run.py

@@ -5,26 +5,23 @@
 # LICENSE file in the root directory of this source tree.
 #
 
+import errno
 import numpy as np
-
-from video_pose.common.arguments import parse_args
+import os
+import sys
 import torch
-
 import torch.nn as nn
 import torch.nn.functional as F
 import torch.optim as optim
-import os
-import sys
-import errno
+from time import time
 
+from video_pose.common.arguments import parse_args
 from video_pose.common.camera import *
-from video_pose.common.model import *
-from video_pose.common.loss import *
 from video_pose.common.generators import ChunkedGenerator, UnchunkedGenerator
-from time import time
+from video_pose.common.loss import *
+from video_pose.common.model import *
 from video_pose.common.utils import deterministic_random
 
-
 def run_video_reconstruction(video_name, compressed_analyzed_path):
     args = parse_args(video_name, compressed_analyzed_path)
     print(args)

backend/video_to_photo.py

@@ -1,15 +1,14 @@
-import os
-import math
-
 import cv2
-from demo_func import run_openpose_for_normal
-from PIL import Image
-from analyse_jump import analyse_npy_side_jump, analyse_npy_side_pingban, analyse_npy_side_juanfu, \
-    analyse_npy_side_gaotaitui, analyse_npy_side_shendun
 import glob
-import time
+import math
+import os
 import os
+import time
+from PIL import Image
 
+from analyse_jump import analyse_npy_side_jump, analyse_npy_side_pingban, analyse_npy_side_juanfu, \
+    analyse_npy_side_gaotaitui, analyse_npy_side_shendun
+from demo_func import run_openpose_for_normal
 
 def split_video(path):
     video_path = path
@@ -42,14 +41,12 @@ def split_video(path):
     print(c)
     return c
 
-
 def process_photo(name, c):
-    print("process_photo"+name)
+    print("process_photo" + name)
     for i in range(1, c):
         run_openpose_for_normal(i, name)
         # run_openpose_for_normal(i)
 
-
 def gather_video(path):
     video_path = path
 
@@ -85,7 +82,6 @@ def gather_video(path):
     out.release()
     im.close()
 
-
 def do_analysis(a_path, video_path, mode):
     print("do_analysis=====")
     print(a_path)
@@ -130,7 +126,6 @@ def do_analysis(a_path, video_path, mode):
         print("kv-over", file=output)
     output.close()
 
-
 def read2(fileName="./cap_file/file1.txt"):
     file = open(fileName)
     db = {}
@@ -152,7 +147,6 @@ def read2(fileName="./cap_file/file1.txt"):
     print("db==== ", db)
     return db
 
-
 def ignore_data_pingban(result):
     result_ig = result
     k = "0"
@@ -207,7 +201,6 @@ def ignore_data_pingban(result):
     print(re)
     return result_ig, re
 
-
 def ignore_data_juanfu(result):
     result_ig = result
     k = "0"
@@ -290,7 +283,6 @@ def ignore_data_juanfu(result):
     print(re)
     return result_ig, re
 
-
 def ignore_data_gaotaitui(result):
     result_ig = result
     k = "0"
@@ -336,7 +328,6 @@ def ignore_data_gaotaitui(result):
 
     return result_ig, re
 
-
 def ignore_data_jump(result):
     result_ig = result
     k = "0"
@@ -414,7 +405,6 @@ def ignore_data_jump(result):
 
     return result_ig, re
 
-
 def ignore_data_shendun(result):
     result_ig = result
     k = "0"
@@ -442,7 +432,6 @@ def ignore_data_shendun(result):
     # 1最大的深蹲角度
     return result_ig, re
 
-
 def write1(result, name):
     output = open(name, "w")
     for re in result:
@@ -453,7 +442,6 @@ def write1(result, name):
     output.close()
     return name
 
-
 def run(video_path, mode, need_split):
     path = video_path
     os.path.split(path)
@@ -482,18 +470,17 @@ def run(video_path, mode, need_split):
     # write1()
     return re
 
-
 if __name__ == '__main__':
     video_path = "./video/43.mp4"
-    path=video_path
+    path = video_path
     os.path.split(path)
-    tur=os.path.split(path)
-    name=tur[len(tur)-1].split('.')[0]
+    tur = os.path.split(path)
+    name = tur[len(tur) - 1].split('.')[0]
 
-    c=split_video(video_path)
-    process_photo(name,c)
+    c = split_video(video_path)
+    process_photo(name, c)
     gather_video(video_path)
-    do_analysis("./capture_image/"+name+"/capture_image{}-1.png.npy",video_path,"shendun")
+    do_analysis("./capture_image/" + name + "/capture_image{}-1.png.npy", video_path, "shendun")
     # result,re=ignore_data_shendun(read2("./cap_file/"+name+"/file1.txt"))
     # write1(result,"./cap_file/"+name+"/file3.txt")
     # write1(re,"./cap_file/"+name+"-file.txt")

frontend/src/util/request.js

@@ -1,5 +1,5 @@
-export const requestUrl = 'http://106.15.1.178:8090/'
-export const httpUrl = 'http://106.15.1.178:8001'
+export const requestUrl = '127.0.0.1:'
+export const httpUrl = '127.0.0.1'
 export default{
     requestUrl,
     httpUrl