Browsing by Author "Keskin, Cem."

Now showing 1 - 2 of 2
  • Thumbnail Image
    Item
    Generative vs. discriminative models for vision based hand gesture recognition
    (Thesis (Ph.D.) - Bogazici University. Institute for Graduate Studies in Science and Engineering, 2017., 2017.) Keskin, Cem.; Akarun, Lale.
    In this thesis, we focus on the problem of modelling sequential data, and particularly hand gestures. We approach the modelling problem using automata theory and theory of formal languages, which allows us to determine the crucial aspects of hand gestures. Furthermore, we show how this approach can help us assess the capabilities of candidate models. The resulting framework can identify problems of models, and set requirements for models to properly represent the gestures. We use this approach to examine common graphical models such as hidden Markov models (HMM), input-output HMMs, explicit duration models, hidden conditional random elds, and hidden semi Markov models (HSMM). We also devise an e cient variant of HSMMs that conforms to all of the requirements set by our previous analysis. We further show that mixtures of left-right models is the most suitable setting for gestures. Finally, we compare all the mentioned models and report the results. In the second part of the thesis, we focus on modelling hand shape with randomized decision forests (RDF). In particular, we extend a known body pose estimation method to hand pose, and then introduce a novel RDF that directly estimates the hand shape. Furthermore, we propose a multi-layered expert network consisting of RDFs that either considerably increases the accuracy, or reduces memory requirements without sacri cing accuracy.
  • Thumbnail Image
    Item
    Vision based real-time continuous 3D hand gesture recognition interface for generic applications based on input-output hidden Markov models
    (Thesis (M.S.)-Bogazici University. Institute for Graduate Studies in Science and Engineering, 2006., 2006.) Keskin, Cem.; Akarun, Lale.
    This study focuses on the application of InputOutput Hidden Markov Models (IOHMM) to the recognition of 3D hand gestures that involve both hand motion and hand posture in a stereo visionbased approach. The proposed system is designed as a real time gestural interface that allows both communicative and manipulative gestures to control target PC applications. The system allows training of new communicative gestures and automatically distinguishes these from manipulative gestures in continuous streams. Uniquely colored gloves with no preset colors are used as markers, which increases the recognition rate and simplifies the hand localization problem. Camera calibration and gesture training tools are provided with the system. The hand shape is modeled with Hu moments and the angle of the hand. As a novel approach the inclusion of normalized time information as an input to the IOHMMs is proposed, which has the effect of a continuous state variable and is shown to handle temporal information better. Another novel method is proposed for gesture spotting in IOHMMbased frameworks, which distinguishes the meaningful gestures in a continuous input stream in real time using a threshold model with a single hidden state. The system is tested on two datasets that have 10 gestures with distinct trajectories, and 20 gestures in 10 pairs that share the same trajectories. The proposed system is able to attain a recognition rate of 97.6% on the former dataset in 2885 trials, and a recognition rate of 94.1% on the latter dataset in 8675 trials. The gesture segmentation test results for the same datasets with the inclusion of the proposed threshold model are 97.1% for the former and 93.8% for the latter dataset.