Face Shape and Reflectance Acquisition using a Multispectral Light Stage

Visit my MSc Research page for more details on my MSc research project. The major contributions of my MSc research are:

• Design and Calibration of Multispectral Light Stage : We have described the design and calibration (geometric and radiometric) of an extended version of the original Light Stage. Our Light Stage design consists of a “beam splitter” based setup that allows simultaneous capture of parallel and cross polarised images. Furthermore, our capture device uses a filter wheel containing narrowband optical filters, to separately record reflectance in different bands of the visible spectrum.

• Minimal Image Sets for Robust Spherical Gradient Photometric Stereo : We extend the analysis of original spherical gradient photometric stereo developed by Ma et al. 2007to consider the effect of deformed diffuse lobes on the quality of recovered surface normals. Based on our modified radiance equations, we explore a Quadratic Programming (QP) approach to correction of surface normals recovered using existing spherical gradient photometric stereo methods. Using the same set of equations, we propose that a minimal set of 4 images can recover surface normals of the quality provided by the existing 6 image method. This minimal image set method has also been described in the following publication:
• Abhishek Dutta and William A. P. Smith. Minimal image sets for robust spherical gradient photometric stereo. In ACM SIGGRAPH ASIA 2010 Sketches, SA 10, pages 22:122:2, New York, NY, USA, 2010. ACM. [PDF, Representative Image, Video (MP4, YT), SIGGRAPH Asia 2010 Presentation Slides, DOI]

• Novel Capture Sequence for Real Time Performance Capture :  Based on our “Minimal Image Sets” analysis and building on the work of Wilson et al. 2010, we propose a new image capture sequence for facial performance geometry capture during dynamic performance. In addition to reducing the data capture requirement of Wilson et al. performance capture framework, the proposed capture sequence also reduces its computational cost by requiring alignment of only one, instead of three, pair of gradient and complement gradient images.

• Stimulus Image Dataset for Psychology Experiment : We explore, for the first time, the use of images captured in a Light Stage to generate stimulus images for a psychology experiment. For a given face, we generate three stimulus images: the first
  contains only the 3D shape information, the second contains only 2D skin reflectance (texture) information and the third contains both shape and texture information. This image dataset has been used for studying the neural representation of 3D shape and texture of a human face.

Project Members : Abhishek Dutta, Dr. William Smith(MSc Research Supervisor)

Real Time Face Tracking and Recognition – rtftr

• Real Time Face Tracking and Recognition (RTFTR) is an open source project that can track and recognise human faces in a video stream.
• RTFTR was used as a platform to study the effectiveness of multiple face detection and recognition algorithms working in unison.
• We also created the IOE Face Database which contains photographs of 13 subjects with around 12 pose variations for each subject. This face database was used to test the performance RTFTR.
• We implemented the following algorithms for face detection and recognition:
  • Face Detection
    • Neural Network Based Face Detection (codename: rowleynn)
    • AdaBoost Algorithm and Cascaded Detector (codename: adaboost)
  • Face Recognition
    • Subspace Linear Discriminant Analysis (codename: lda)
    • Face Recognition using Gabor Wavelet Transform (codename: kepenekci)
• Version 0.1 of RTFTR was submitted as the final year project for our undergradute engineering degree at the Institute of Engineering – Pulchowk Campus.
• Adding gtk based GUI and with some performance improvements due to OpenMP, we released version 0.2 and participated in Open Software Challenge Nepal (OSCN) 2009. Our project was awarded the second prize in this national software competition with more than 70 competing projects.
• RTFTR has been downloaded 3,638 times from 83 countries.

Awards: (2nd prize) Open Software Challenge Nepal (OSCN) 2009

Project Members : Abhishek Dutta, Anjan Nepal, Bibek Shrestha, Lakesh Kansakar

Project Supervisors : Dr. Jyoti Tandukar, Sharad K. Ghimire

Intelligent Multiplayer Snake Game – iSnake

• Intelligent Multiplayer Snake – iSnake is a multiplayer snake game created in Java.
• We used Java SWING to create the user interface and Apache MINA was used for the multiplayer functionality.
• The initial version of iSnake was submitted as the 3rd year minor project of our undergraduate engineering degree at the Institute of Engineering – Pulchowk Campus.
• We implemented two automatic path computing algorithms to drive the computer controlled snake. One of the algorithms (codenamed viper) was so intelligent that we had to reduce its capabilities after a lot of our players complained of the game being very hard

Awards: (2nd prize) Open Software Competition – LOCUS 2007/08  , (1st prize) Inter College Undergraduate Project Competition – KUCC Software Meet 2008

Project Members : Abhishek Dutta, Jitendra Harlalka, Suraj Sapkota

Unofficial archive of Institute of Engineering – uIOE

• I started this project to create a digital archive of all the projects completed at the Institute of Engineering – Pulchowk Campus.
• When I was planning my final year project, I felt the need for such a digital archive. At that time, our college had such an archive only in hard copy format in the library.
• This project is now being maintained by a current 4th year student (3rd generation maintainer) at the Pulchowk Campus.

Project Members : Abhishek Dutta

iRobot

• In this project, we created an animation of a robot (made up of rectangles) walking along a fixed path.
• We submitted this project, as the project for our 3rd year Computer Graphics module.
• Camera viewpoint, Walking Speed, Zoom Level can be changed using the simple controls provided in the Java GUI.

Project Members : Abhishek Dutta, Suraj Sapkota

Nokia 6100 LCD support in serdisplib project

• I contributed code to add support for Nokia 6100 color LCD in the serdisplib project. In fact, this was my first contribution to the open source world.
• I illustrated the use of mobile phone LCD interfacing using a simple python application called “gmail2lcd”. This application displayed alert on the mobile phone LCD on receipt of new email.
• An article titled “Using ‘serdisplib’ library to drive mobile phone LCD” was published in the “Do-It-Yourself” section of Electronics For You (May 2008 issue) magazine. This was the first time when I was paid money for writing an article. I have been reading this magazine since I was 12 and was delighted to see my name in one of the articles of this magazine.

Project Member : Abhishek Dutta

A tribute to Donald W. Davies

• We participated in the IEEE CHC61 web design competition with this website. The theme of this competition was “Unsung Heroes”.
• This website pays tribute to Donald W. Davies who was responsible for one of the fundamental concepts that enables today’s advanced computer networking systems: digital packet-switching.
• The result of the competition was announced on Aug. 30, 2007. Unfortunately, our entry did not make into the TOP 10 list of finalists.
• Note : It seems that I have lost the background images required to display this website correctly. See this image for a screenshot of original website design.

Project Members : Abhishek Dutta, Anjan Nepal, Lakesh Kansakar

8-bit microprocessor design (based on SAP-1 architecture) using Verilog

• We created a VERILOG model of a computer based on SAP-1 (Simple As Possible) architecture.
• The final lab assignment of our 5th semester module “Computer Architecutre and Design (CAD)” required development of this SAP-1 computer VERILOG model.
• SAP-1 (Simple As Possible) architecture was introduced by Albert Paul Malvino in his book Digital Computer Electronics – An Introduction to Microcomputers: SAP-1(Page 140). Tata McGraw-Hill Publishing Company Limited 1990.
• Project Files
  • Complete VERILOG Source Code : sap1_verilog_code_zip.pdf (rename the file to sap1_verilog_code.zip)
  • Verilog Project Code Listing : Code.odt or Code.pdf
  • Project Report : CAD-SAPreport.odt or CAD-SAPreport.pdf
  • Logic Analyser Trace : Resultpage.odt or Resultpage.pdf

Project Members : Abhishek Dutta, Bibek Shrestha

Interfacing with the ISA Bus

• We did this project while exploring the wonderful world of hobby electronics.
• We designed an interfacing circuit to access the I/O lines of Industry Standard Architecture (ISA) bus available in some old motherboards. This provided us with 32 general purpose I/O lines.
• An article describing this project was published in the March 2006 issue of the linuxgazette magazine.

Project Members : Abhishek Dutta, Badri Adhikari, Pushpeshwor Pokharel

kinbech@ioelive – ebay like auction site

• We created a php-mysql based “ebay” like auction website called kinbech (a Nepali word for “buy and sell” ) for the students at Institute of Engineering-Pulchowk Campus
• It allowed students to buy, sell and request items like books, notes, CDs, etc
• kinech@ioelive was launched on Oct. 03, 2005. However, kinbech was not widely used probably because it lacked online payment system and robust delivery scheme.

Awards : (2nd prize) Web Design Competition – LOCUS 2005

Project Members : Abhishek Dutta, Bibek Shrestha