Within this chapter we reviewed the background literature and geometric properties of projective geometry that are essential for understanding the remaining material covered in this thesis. We showed how planar and 3D primitives such as points, lines, planes and conics are represented and how they are transformed under projective imaging. We gave an overview of the projection process of worldspace 3D structure into the image, and described both the simple linear camera model and more complex cameras with radial lens distortion. We outlined the epipolar geometry and correspondence conditions between points in two views and also considered the specialized auto-epipolar case which arises from a linearly translating camera with fixed internal parameters. We then reviewed the literature on camera calibration and subsequently the back projection and triangulation of worldspace structure.
Worldspace structure commonly occurs within single planes. Subsequently, projection of planar worldspace structure into the image has a simpler form. The transformation of structure from one plane to another is achieved by a 3×3 matrix mapping known as a homography. We reviewed the mathematics and essential properties of planar geometry then gave details of some commonly occurring specialized planar transformations, such as image mosaicing and repetition of planar patterns.
The homography transformation mapping one point set to another can be determined from the known correspondences of points between the two planes by using a direct linear transformation technique. On the other hand, we may not have physical point correspondences between the canonical reference plane and the image plane. We may though have some additional knowledge about the structure on the reference plane that allows us to employ a stratified approach to recover the metric structure.
Geometric properties of the plane can be classified into three main groups of transfor- mation. Further details and properties of these classes of transformation can be found within the discussion in appendix A.1.
• Perspective transformations. Parallelism and orthogonality of lines are not preserved. Length ratios on lines are not preserved. The ideal points and the line
at infinity become finite after transformation.
• Affine transformations. Parallelism of lines is preserved, although due to skew, orthogonality is not. Ratios of lengths on parallel lines are preserved. The set of ideal points remain ideal, though are not fixed pointwise by the transformation. The line at infinity remains fixed.
• Similarity transformations. The circular points I,J, angles between lines and ratios of lengths all remain invariant under transformation.
Metric properties of the scene plane are recovered by determining the homography map- ping, formed from the known scene plane structure constraints, that maps the imaged vanishing line and circular points back to their canonical positions. Each stage within the stratified rectification process is designed to remove a number of degrees of freedom from the eight required to compute the planar homography.
The set of ideal points all lie on the ideal line, thus identification of the imaged vanishing line of a plane allows us to compute the perspective transformation Hp that recovers the affine properties of the plane.
The circular pointsI,Jof the ideal plane encode the Euclidean coordinate axes within a single complex conjugate entity. Constraints can be formed on the circular points from prior knowledge of the scene structure, and enables us to compute the affine transfor- mationHa that recovers the metric properties of the plane.
Since the circular points I,J remain invariant to similarity transforms, further con- straints that recover structure within a common coordinate system must be explicitly chosen by the experimenter. A similarity transformationHs may be chosen to place a known point at the origin, scale a common feature to unit length or align a known vector with one of the coordinate system axes. The combined set of stratified transformations H=HsHaHp then recovers the required properties of the metric plane.
We hypothesize that articulated limb motion within human gait is approximately planar. Almost all of the perceived limb motion is contained within a single plane. Consequently, gait has sufficient properties that allows us to exploit the structure of planar articulated limb motion in order to recover the fronto-parallel motion dynamics, with no prior knowl- edge of the camera calibration. As an example, the stratified reconstruction technique was applied to a synthesized image of an obliquely viewed motion figure pattern in order to demonstrate that the canonical fronto-parallel view could be successfully recovered. Our review of the camera calibration literature showed that a planar calibration target could be used to determine the intrinsic parameters of the camera. Two or more different orientations of the calibration target are sufficient to compute the camera parameters. A similar argument can be made for the motion of gait. Since we can recover the homog- raphy that transforms the fronto-parallel reference plane of subject motion to its image
then a minimum of two different imaged motion trajectories are sufficient to determine the intrinsic parameters of the camera. The work presented in this chapter provides the groundwork for what is essentially an “Auto-calibration from gait” algorithm (a close second candidate choice for the title of this thesis).
The later chapters of this thesis are then concerned with determining a suitable model representation of articulated limb motion, identifying the static features of gait that may serve as a useful biometric, and a practical validation of the theoretical auto-calibration method identified within this chapter.
Static Features of Human Gait
3.1
Introduction
Within the context of human identification, subject gaits can be observed within various situations and from different viewpoints. Viewpoint and environment can be carefully controlled within a laboratory setting, while little or no control is possible for outdoor scenes [12, 54]. Many features are proposed in the literature for gait recognition in- cluding optical flow, joint angles, silhouette, etc. We can categorize them as static and dynamic features that evolve in time. Static features reflect instantaneous, geometry- based measurements such as stride length / cadence, limb lengths and height [54, 6, 7]. Dynamic measurements are sensitive to the temporal motion structure of subject activ- ity, such as joint angles, optical flow, symmetry and self similarity [79, 20, 101, 23, 43]. Normal walking conditions (constant and natural walking speed, carrying no objects, level ground plane, etc.) are some of the fundamental assumptions made in most current techniques. Many proposed features and algorithms do not work well if these conditions are violated. Even though gait patterns are repeatable most of the time, changes in walking conditions affect these motion patterns. There are many factors, both physical and psychological, within our daily lives that can influence the variations between our motion patterns such as walking speed, cadence, ground surface, load carrying and state of mind. Understanding the characteristics of gait motion patterns under various walk- ing conditions will help improve the techniques used in further gait research. Here, we are interested in human gaits across different speeds. In particular, we want to under- stand the patterns of gait motion parameters such as stride length and cadence, which are potentially measurable by computer vision techniques.
Bobick and Johnson [8, 54] develop a gait-recognition method that recovers static body and stride parameters of subjects as they walk. Their technique does not directly anal- yse the dynamic gait patterns, but uses the action of walking to extract relative body parameters. The set of static body parameters measured are four lengths: the vertical
distance between the head and foot, the distance between the head and pelvis, the dis- tance between the foot and pelvis, and the distance between left and right feet. These distances are measured only at the maximal separation points of the feet during double support phases of the gait cycle. However, they consider step length by itself to be a static gait parameter, while in fact it varies considerably for any one individual over the range of walking speeds. The typical range of variation in step length for adults is about 30 cm [50], which is far from negligible. Their method for estimating the separation dis- tance between feet does not exploit the periodicity of walking, and hence is not robust to tracking and calibration errors.
Davis and Taylor [24] develop an approach for recognizing human walking movements using low level motion regularities and constraints (gait period, stance/swing ratio and double support time). Biomechanical features for classification are automatically ex- tracted from video sequences of walkers. A multiplicative classification rule using statis- tical distances is then used to determine whether an unknown motion is consistent with normal walking patterns.
BenAbdelkader and Cutler [5, 6] develop a correspondence free method to automati- cally estimate the spatio-temporal parameters of gait (stride length and cadence) of a walking person from imaged motion. Cadence is estimated using the periodicity of a walking person. Using a calibrated camera system and a known ground plane, the stride length is estimated by first tracking the person and estimating their distance travelled over a period of time. By counting the number of steps and assuming constant velocity during walking they are able to estimate the stride to within 1 cm for a typical out- door surveillance configuration. They show that stride length and cadence are linearly related over a range of gait speeds. Their approach works with low-resolution images of people, is view-invariant, and robust to changes in lighting, clothing, and tracking er- rors. It achieves its accuracy by exploiting the nature of human walking, and computing parameters of stride and cadence over many steps.
Further work by Tanawongsuwan and Bobick [103, 102] explores the spatio-temporal gait parameters (stride length and cadence) across a number of controlled walking speeds. They give an in depth study of 15 people, with repeated measurements on different days, for motion obtained from treadmill walking. Their results agree closely with the findings of BenAbdelkader and Cutler.
We understand that the prior work by both BenAbdelkader and Tanawongsuwan has answered many of the relevant questions with regard to the parameters of stride length and cadence. Both their techniques require a calibrated camera and that the ground plane is known, in order to extract the required features for recognition. Our research differs from theirs, since we are interested in observing the dynamic behaviour of the articulated limb angle motion over a range of controlled walking speeds. We propose a suitable motion model that enables us to extract the features of gait without the need
to know the camera calibration, ground plane or subject trajectory. In this chapter, we outline a suitable representation to model the articulated limb motion and give a brief analysis for a small set of trial subjects. In essence, we provide enough information to justify our choice of suitable gait features and biometric representation of motion, that facilitates further development of the view invariant reconstruction methods described within later chapters. Although our set of gait features and motion model function differ from the work of BenAbdelkader and Tanawongsuwan, we show that the results are proportionally similar and correspond well with their findings.
We first give an overview of the terminology and biomechanics of subject motion from the medical literature [50, 84, 15, 77], in order to better understand the nature of gait. We describe the sequence of events within a gait cycle that allows a person to progress forward. Eight individual phases of gait have been identified, each with a different functional objective, that form six distinct motion patterns known as the determinants of gait.
Literature within the context of planar geometry [62] indicates that constraints can be formed from known ratios of lengths within an imaged scene plane in order to determine the transformation that recovers the true metric structure (angles and length ratios). We then hypothesize that articulated limb motion is approximately planar and proceed to verify this assumption by measuring the deviation of 3D joint positions from pla- narity. We give a brief outline of various motion marker systems that other researchers use in order to capture the dynamics of gait motion, then describe the marker system and experimental set-up used within our laboratory for the purpose of these experi- ments. The worldspace joint positions are computed by triangulation of imaged point correspondences over a number of camera views. We then give an analysis of the level of pixel reprojection error between the model and corresponding image measurements, caused by the limb swing plane assumption, to determine the validity of the proposed model.
The human skeletal structure is articulated but with fixed length limb segments. These limb segments provide a set of static parameters of gait which remain constant over the entire image sequence. We model the articulated limb motion with a suitable periodic function, hence motion parameterisation is determined from all available data within the image sequence. Techniques that look for specific key frames [8, 54], i.e. positions of maximal foot separation, may be susceptible to lost or occluded frames. The proposed limb motion model is robust to both noise and missing data.
Finally, we analyse the behaviour of the motion parameters over a range of controlled gait speeds for a small trial set of subjects. We emphasize the parameter properties that remain invariant over these speeds and outline a biometric feature vector suitable for recognition purposes. We show similar results to the works of BenAbdelkader and Tanawongsuwan. Finally, a brief discussion on the major sources of error and further
areas of development are addressed.