The Bradford Multi-Modal Gait Database: Gateway to Using Static Measurements to Create a Dynamic Gait Signature

Aims: To create a gait database with optimum accuracy of joint rotational data and an accu-rate representation of 3D volume, and explore the potential of using the database in studying the relationship between static and dynamic features of a human’s gait. Study Design: The study collected the motion capture were used as dynamic features. The Pearson correlation coefficient is used to evaluate the relationship between the two sets of data. Results: A new database was created with 38 subjects conducting four forms of gait (walk, run, walk to run, and walking with a hand bag). Each subject recording included a total of 8 samples of each form of gait, and a 3D point cloud (representing the 3D volume of the subject). Using a P-value (P<.05) as a criterion for statistical significance, 386 pairs of features displayed a strong relationship. Conclusion: A novel database available to the scientific community has been created. The database can be used as an ideal benchmark to apply gait recognition techniques, and based on the correlation analysis, can offer a detailed perspective of the dynamics of gait and its relationship to volume. Further research in the relationship between static and dynamic features can contribute to the field of biomechanical analysis, use of biometrics in forensic applications, and 3D virtual walk simulation.


INTRODUCTION
There are many gait databases available for the public; most prominent are the USF HumanID gait challenge database [1], and the CASIA gait database [2]. They both provide abundant 2D video data of walking subjects with different variants (Clothing, shoes, surface, and angle of surface being walked upon; either incline or uphill). A lot of gait recognition related databases emerged following DARPA's HumanID at a Distance program. Other databases also include the University of Southampton's 3D Gait Database [3], the Carnegie Mellon University's MoBo database [4], the HUMABIO database [5]. Although there are a lot of databases available for gait recognition, none of them can provide an accurate representation of joint movement and rotation along with an accurate representation of the physical 3D human body.
Therefore; the core of this work is to develop a new gait database using motion capture and 3D laser scanning technology. The motion capture data would provide the accurate dynamics of a walk, while the 3D laser scanner would provide the accurate 3D physical representation of the human body. This kind of accuracy would allow the further study of the relationship between body's physical composition (size, height, build) and gait dynamics. This paper will discuss the covariants and modalities in current databases, an insight into the creation of the BMMGait Database. It will also discuss the results of a correlation analysis of the two sets of features to explore the potential of using static features as a basis for predicting dynamic features.

COVARIANTS AND MODALTIES
Gait has many variables that can affect its cycle. Several variants have been addressed by the USF HumanID gait challenge database which include: Camera angle, shoe type, walking surface, briefcase carrying, and time passage between recordings [1]. Other databases have explored other variants which include: Clothing type, gait speed, pitch or slant of surface, load and weight (Backpack, carrying a ball…etc.). [2][3][4][5].
Even though databases explore the different variants that influence a gait, there is a need to investigate the different mediums used to record a gait. The most common devices used are cameras, even though they can vary in resolution, size, lens type, and quality. There are other databases which include recordings using infrared cameras, multiple cameras setup, motion capture, Doppler low freq. devices [2,3].

BMMGAIT DATABASE
The database was created to develop a multimodal gait database that can be used as a benchmark to apply various gait recognition experiments and techniques. The database's current modalities are:

Motion Capture
The motion capture system used in this database consists of 16 Vicon T20 cameras. These cameras offer a resolution of 2 megapixels that capture 10-bit grayscale images at 500 frames per second. The motion capture in our experiments was shot at a resolution of 2 MP and at a frequency of 100 Hz. The cameras and motion capture process is managed and controlled by Vicon Blade. Blade provides the control and management of actor (subject) setup, recording the motion capture data, and clean up.
Marker setup is the manner in which the white reflective markers are placed on a subject. The 57 marker setup used in this database is the standard used at the University of Bradford motion capture studio, which is usually intended for real-time 3D simulation for the fields of entertainment and video games.

Video Camera
The subject's gait was captured using three cameras. Each camera was placed on a tripod. One camera was placed parallel to the walk direction in order to capture a side view of the walk. The second camera was place in the corner of the studio to capture a 45 degree angled view, similar to a closed-circuit television (CCTV) camera's field of view.
The cameras used in this database were the Canon EOS 5D Mark II. The video recorded was of a full HD resolution (1920 x 1080), at 25 frames per second.

3D Laser Scanner
A Faro Laser Scanner Photon 120 3D laser scanner is used in this database. It scans a 360 degree horizontal field of view with a speed of 120,000 points per second. In this setup, the laser scanner was controlled and the recording was managed through the use of Faro Scene. The scanner was used to take four scans of the subject: frontal, posterior, and the left and right sides. Scans took place before the motion capture recording started. Although there was a very minimal risk of the scanner laser being directed at a subject's eye, precautions were considered by the use of safety goggles.

Subjects
The main objective of the database is to provide one unified database that includes different modalities in regards to recording media used. In this database, every gait sequence is available in one of the following three formats: 1-Video recording of a subject parallel to the camera's recording plane 2-Two alternative video recordings of the subject (frontal, and angled)

3-Motion capture data (3D motion data)
Accompanying the motion data formats will be two 3D point cloud (3D measurement data) datasets: 1-3D scan of room 2-3D scan of the participant Since the use of treadmills is debatable [6], it was decided not to use them in this database and rely on the length of motion capture studio. The current database includes 38 participants. Each participant was asked to wear the motion capture suit. The markers were placed as close as possible to their targeted joint placement. Furthermore, the suits were adjusted to be as tight as possible without limiting movement, to avoid any movement of the material. Although minimal sliding of the material is unavoidable, yet it is a common assumption in such methods, and can be factored out by the motion solvers within the Vicon blade software. The subjects then were asked to conduct the following actions in the chronological order within estimated 1 hour duration: 1-Conduct a walk 8 times across the room 2-Conduct a run 8 times across the room 3-Conduct a walk to run transition 8 times 4-Conduct a walk carrying a bag using the left arm 8 times 5-Conduct a walk carrying a bag using the right arm 8 times In all of the actions mentioned above, the subjects started from a stationary position. They would walk/run at their own comfortable speed, in a straight line across the room, which was of a length of 8 meters. Once they traveled that distance, they would then travel back across the same line in the same manner.
The initial subjects were contacted through the use of flyers within the Centre for Visual Computing and the School of Engineering and Informatics at the University of Bradford. Each volunteer was handed an information sheet about the database and the process of recording. They were each requested to sign a consent form before any recording session takes place. The first phase of this database included the recording of 38 subjects who had no previous injuries or surgeries that would influence their gait.
Each volunteer of the thirty eight recorded subjects was handed an information sheet about the database and the process of recording. Those subjects consisted of 5 females and 33 males with an average age of 28 years old ranging between 19 and 45. The Average weight was 76.1 Kilograms, ranging between 45 and 130 kilograms. The average height was 173.6 centimetres, with a range between 156 and 190 centimetres.

DATA PROCESSING AND ANALYSIS
Each data type followed a unified procedure to convert the raw data into useable data for further analysis and testing.

Video Data
With regards to the video data, it was required to be: 1-Classified and cut into separate full gait cycles and; 2-Processed manually to track dynamic features.
The data recorded in each session was shot continuously, which means; for each subject, all forms of gait are included in one continuous video file. Therefore there is a need to cut the video into sections according to their form (walk, run, walk to run transition, walking with a bag). Each video was cut into one complete gait cycle, starting with the left heel strike. The start and end of each cycle was manually selected to start with the heel strike, which coincides with the maximal elevation of the thigh in the cycle. Video tracking software was used to track the different features of a subject's gait. Specifically, Pixel Farm's PFTrack (version 5.0) was used to divide video tracks while tracking the required joints needed to process the dynamic gait features.
Once the video is divided in accordance to its sequence number and form, tracking of key joints was executed on the subject using both; PFtrack's automated tracking tool and manual tracking by user input. Those joints that were tracked included: mid-section of the hip, left and right knee, left and right ankle, left and right ball of the feet, left and right feet tip, left and right shoulders, left and right elbows, left and right wrist, and finally the top tip of the head.
Finally, the tracked data is exported as individual files that represent vertical (X) and horizontal (Y) positions of the tracked points. All files are saved using the following naming convention:

Sub_####_dat_xxx_yy_o_Joint_A
Where #### is the subject ID, xxx is the gait form, yy is sequence number, o is the left (L) or right (R) side of the body, Joint is the name of the joint being tracked, and A is the axis (X or Y). Therefore, subject #1's X-axis tracking of the right knee when the subject conducts his/her first walk sample is named as: Sub_0001_dat_wlk_01_R_knee_X.txt

Motion Capture
For the motion capture data to be usable, it must be converted to either positional (P) data in <Px,Py,Pz> or rotational(R) data <Rx,Ry,Rz>. Therefore a reconstruction of the human skeleton is required, and is processed through the use of Vicon Blade (version 1.7.0).
The data is first processed for what is called ROM (Range of motion), in which the range of motion of the subject is identified. This is followed by a calibration, in which the marker skeleton (a virtual 3D representation of a subject skeletal model) is adjusted according to the subject's body size. Finally, the new skeleton is used as a base for solving all the gait samples, which outputs rotational values for all joints available in the used marker setup. The data is then exported as an ASCII file containing the rotational data of all joints and saved according to the following naming convention:

Sub_####_dat_mcp_xxx_yy_.txt
Where #### is the subject ID, xxx is the gait form, and yy is sequence number.

3D Laser Scan
The aim of capturing 3D scans of the subject has two reasons: 1. To be able to accurately provide scalar data in regards to 2D measurements of the human body (The list of measurements is illustrated in Fig. 1). 2. To be able to study the body from a 3D point of view (volume, surface distribution, etc…) In regards to the first aim, direct measurements using the point measuring tool in Faro Scene is utilized.

Fig. 1. An illustration of the 2D measurements extracted
The four scans taken of every subject were conducted separately. First, a front scan was taken, followed by the right side, the back, and the left side. To maintain the same pose between scans, placement points for the feet were used, as well as a defining the position of the arm through the use of two chairs (the subjects would rest the tip of their finger on the chair to maintain stability).
Although there were minimal movements between scans, yet it provided a more accurate measure of volume than the use of volume estimating algorithms from single camera sources. For the purposes of studying the body in 3D space, the four separate scans from the Faro LS laser scanner were merged together. Because there was very minimal overlap between the two scans of every subject, it was required for this step to be executed manually.
The process involved creating a 3D surface from the point cloud (captured by the laser scanner) using InnovMetric Polyworks (version 10), which were then exported in the OBJ format. The two separate files were imported into Autodesk Maya (2011 version), where they were manually merged and placed together. The final step was best fitting a generic 3D human mesh to the merged 3D scans. The resultant files were saved in the /inf folder using the following naming convention: Sub_####_inf_3dp.obj where #### is the subject ID number.
An example of a 3d mesh of a subject is shown in Fig. 2.

DATA PROCESSING AND FEATURE EXTRACTION
In order to demonstrate the application of this novel database we perform a correlation study usingfeatures that are common in gait recognition applications and include both dynamic and static features. In this correlation study, phase-weighted magnitude (PWM) of the different joint rotations of a subject was used as a dynamic feature. The method is driven from a technique developed by Caundo et al. [7]. Based on the outcome of a study by Chew Yean et al. [8], in most gait samples the magnitude spectrum produced by a Fourier Transform algorithm converge to a zero value beyond the fifth harmonic.
In the same work it was also proven that a phase weighted magnitude in which the phase component is multiplied by the magnitude component provides stronger discriminatory potential than the use of the phase or magnitude component independently. In the mentioned study [8], only the first two harmonics in the thigh rotational data and the first three harmonics in the lower leg rotational data were used because of their highly discriminative properties. The same method was used in later studies and applied to both 2D and 3D models [7]. Therefore, only the 2nd to 5th components were used in the analysis to avoid noise and irrelevant data.
With most gait recognition techniques, the static features are extracted from the 2D or 3D model used to describe the subject's gait [3]. Computer vision based static features extraction techniques applied to video or two dimensional images carry a considerable amount of error, therefore; in order to acquire data that is accurate, a reconstruction of the subject's three dimensional body volume was created from the four 3D laser scans using Geomagic Polyworks to reverse engineer the point cloud to a 3D surface. Autodesk Maya is then used to combine multiple meshes. The choice of features and the manner in which the subject's 3D volume was divided was based on logical physical landmarks as well as the discriminative static features mentioned in the study by Baofeng et al. [9] as well as in the study by Alawar et al. [10]. Each individual part's volume and surface area were then calculated using Autodesk Maya's MEL commands ('computePolysetVolume' and 'polyEvaluatearea'). There were a total of 42 3D static features used. A visual representation of the division map of the body is shown in Fig. 2.

ANALYSIS AND RESULTS
The correlation coefficient was used to examine the relationship between the static and dynamic features. The correlation coefficient matrix is the covariance, features extracted. The covariance calculated using the following formula Where is the expected value average.
Based on the study by Alawar et al. features with (P<0.05) was considered significant. When such a threshold the current study, 386 pairs expressed significant correlation. purpose of further analysis and detailed the highest correlated features, the (-0.9 > r or r > +0.9) was used. threshold resulted in 119 pairs of correlated features. The highest feature pairs are listed in Table 1.

DISCUSSION
The findings by Alawar et al. [10] are these results, and furthermore, provide detailed insight into the contribution individual rotational axis in a correlation between the joint and static    [11] describe the effect carriage on the temporal relations the trunk and the leg. Another study et al. [12] describes the contribution movement to the reaction moment ground. The study compared a which arm movement was restricted, found to directly contribute to greater moment from the ground, hence human body to adapt and increase expenditure and muscle usage. motion of the arms directly contributes effort of the leg during gait. David conducted a study on the effect of on static posture and gait dynamics. a direct influence of an increase weight in the upper extremities (carrying on gait dynamics relating to lower such as stride length and frequency.

CONCLUSION AND FUTURE
The Bradford Multi-Modal Gait (BMMGD) is a novel database. Although are databases that contain motion the BMMGD contains accurate 3D the subjects alongside 2D multi footage. We give an example of might be used by performing an correlation study which exhibits strong between 386 pairs of features with 119 pairs when (r<-0.9 or r>0.9).
; Article no. BJAST.13426 For example, the showed a significant right leg. Such relate weight and gait kinematics. effect of load on relationship between study by Collins contribution of arm moment from the gait cycle in restricted, and was greater reaction hence requiring the increase energy Therefore, the contributes to the David et al. [13] carrying a bag dynamics. It describes increase in size and (carrying a bag) lower extremities frequency.

Gait Database
Although there motion capture data, 3D laser scans of multi-view camera how this data an exploratory strong correlation with (P<0.05) and These results