Project meeting minute - 05

Date & time, duration

The meeting was held on 2010-10-29.

Venue

Office room of Dr. Chandana Gamage

Members present

All the members were present there.

Topic of the discussion

Contribution to the project by each member

Brief Discussion of the discussion

As the deadline of the SRS was scheduled to be on 5th of November we discussed the important facts on that and Chandana sir wanted every of us to come up with the contribution for the project during that week. So we told to the sir what we have done on particular areas of the project since we have divided those topics among ourselves beforehand. Lahiru the team leader of our project had come up with his findings on the topic HMM (Hidden Markov Model) along with other details related to the area 2D to 3D mapping. In my case I have done my research on neural networks as a replacement methodology for HMM. And Thusitha and Hasitha did their researches on markerless models and interpretation of 3D virtual environment respectively. Chandana sir appreciated our effort and encourage us do further research on selected areas as we have finalized. For an instance for modeling 3D objects out of 2D model he advised us use neural networks rather than HMM based on the fact of the learning ability of neural networks. More importantly apart from the project related works he asked us to study other subjects also and ask us to study other subjects well also and not to let down them because of the project.

Key topics discussed

1. HMM

2. The good side of artificial neural networks over HMM

3. Approach of marker based models

4. Interpretation of 3D environments

Comments and agreements

Today sir went through the things we did as according to the topics we were assigned and make his feedback on several design decisions and asked us to continue those topics further ahead.

Features and operating environment of BubbleBoy.

BubbleBoy Features

BubbleBoy is a project that targets creating a simple user interface for the user to communicate with computer.  It contains following features,

• Operates with commodities like web camera which is easy to find for the user.
• Achieve all the hard work of program using software and user need not to be troubled about installing difficult hardware.
• Track the full body motion of the user and simulate the computer program accouding to it.
• Provides different kinds of experience (different environments) to user as it helps different users’ choices.   
• Provides a less complicated and low cost product to have an invaluable experience.

Operating Environment of BubbleBoy

BubbleBoy provides a scenery of a natural environment experience through computer screen.And it gives the chance of interaction with computer using full body motion(Actions). Operating environment of the system includes following features.

• Two web cameras for capturing the actions of the user.
• Computer with
• High processing power: Core 2 Duo 2.2 or more
• Secondary storage of 5GB separated for the application.
• System memory of 1GB or more

Sceneries that will be there for the end users

In the project "BubbleBoy"we are hoping to deliver a system which would enable interaction without any specific hardware development, but only with aid of software. Only thing user has to do is to paste the markers on him or her which another technique we have exploited to provide the user with better performance and effective outcome. We need to provide appropriate scene as required by the user. The user will enable to select one of following activities.

· Swimming in an aquarium/sea/river.

· Walking alone by seaside/riverside.

· Walking at a museum

· Drive a racing car

· Walking on moon/space

· Having a walk in a volcanic site

· Tsunami

· Surfing

· Snow board

We can add any kind of scenery since we are hoping to capture the whole body movement. We are hoping to design the graphical environment similar to natural conditions to some extent.

It should be noted that in the process of proving scenery to this application we have focus our attention on physically impaired people who are unable of having these kinds of experience during their real life the apart from the average users. They will definitely enjoy it as our application will be fully enriched with life like experiences.

Sofware Specification

When we go through the research we look at the environment design related to the project. This projects` main target is to interact with the movement of the user in real time. As the project requirement, this interaction should reflect via a high graphical interface. So we look at several choices and did some testing to ensure the ability of available resources.
We consider following choices
• XNA
• Crytec
• JMonkey
XNA
XNA is a framework is developed based on native implementation of .NET Framework 2.0 on Windows. The framework runs on a version of the Common Language Runtime that is optimized for gaming to provide a managed execution environment. The runtime is available for Windows XP, Windows Vista, Windows 7, and Xbox 360. Since XNA games are written for the runtime, they can run on any platform that supports the XNA Framework with minimal or no modification. Games that run on the framework can technically be written in any .NET-compliant language, but only C# in XNA Game Studio Express IDE and all versions of Visual Studio 2008 are officially supported.
JMonkey
jMonkey Engine (jME) is a high-performance 3D game framework, written entirely in Java. OpenGL is supported via LWJGL, with JOGL support in development. For sound, OpenAL is supported. Input via the keyboard, mouse, and other controllers is also supported. The most important thing in this is, it run on Windows, Mac OS, and Linux; true cross-platform thanks to the JavaVM.
Crytect
CryEngine is a high performance 3D game engine developed by crytec and they recently release CryEngine 3 . The new engine is being developed for use on Microsoft Windows, PlayStation 3 and Xbox 360. As for the PC platform, the engine is said to support development in DirectX 9, 10, and 11. It supports and contains rich graphical usages such as water reflection, Flow graph, Real time soft particle system & integrated FX editor, Road & river tools, Vehicle creator, Real time dynamic global illumination, Deferred lighting, Natural lighting & dynamic soft shadows
We do compare above technologies and came up with JME which is platform independent. When consider XNA it supports only to windows and Xbox. And when comparing the maturity levels of the XNA and JME, JME shows much efficient and effective performance. Cry engine is one of greatest 3D game engine in terms of detailing. But it need certain level of hardware requirement and it is in the commercial market. So we look at JME which stand alone in both cases.

Project meeting minute - 04

Date & time, duration

The meeting was held on 2010- 10 -21.

Venue

Office room of Dr. Chandana Gamage

Members present

All the members except for H.G.R Dileepa were present there.

Topic of the discussion

1. 3D model mapping from 2D images

2. Human interpretation techniques (with markers/marker less)

3. Virtual environment

Brief Discussion of the discussion

Dr. Chandana Gamage asked us to do the comparison among the available technologies and come up with simple documents which mention about reasonable explanations and comparison etc. Sir asked us to explain research papers read to the group members because that will help to understand the things easily. While doing the research Sir asked us to note down all the novel ideas we have to discuss and figure it out during the meeting sessions. And also asked us to think about possible new theory for add high value to the project. We are supposed to update the wiki page in moodle about the project progress and also Sir asked us to create website for the project and update it as well. While we discuss within group we were asked to do things to broad the discussion such as put ideas and comments on related blogs and discussion forums, etc. Sir mentioned about disabled soldiers in “Ranaviru sevana” where suitable place to do the demonstration of our project. At the end of the discussion we planned about what we do in next week such as continue our previous work carried out on research and prepare the System requirement Specification document.

Key topics discussed

1. 3 design areas defined under topics under discussion

2. extra things to add value to the project

3. plans for the next week

Comments and agreements

1. explain the research papers each one has gone through to the others.

2. create a web site for the project and update it.

3. continue the previous work with reading research papers.

4. prepare the Software Requirements Specification document.

Project "Bubble Boy" Against the Existing Technologies

 The point at which it differs from the rest of the applications is the way in which the complexity of our application is managed. In existing applications most of the things are done using highly sophisticated hardware either embedded into the computer or placed on the user which in turn would cost a lot. In that scenario role of our application comes into play where it will not be required sophisticated hardware on either case. The hidden complexity will be managed by the software and commodity hardware components which will be easily and affordably available will only be there for sake of the purpose. Another notable fact is the kind of the target audience for this system. The target audience is the average set of people who does not have that much computer literacy. So the majority of them find it uncomfortable to continue with that and all what they want is to feel the experience provided by this system with minimal effort without having to struggle severely understand and use the system. Otherwise they are dissatisfied with the system no matter how sophisticated the system is. Since this system could be developed in order to capture such market, this would be great project people who do not have computer literacy.

This innovative research would consider not only the movement, but also the speed of action. Eventually this gives an interaction which is much natural and user friendly. Person throws a spear and a stone to a river are two different activities. But both activities contain almost similar movement. Only change might be the speed.

Implementation

1.1 Software Development Model

During the project, it requires to follow certain software development model in order to achieve better quality output. Since the requirement would affect with the present technology, it is prefer to use Iterative model which is much dynamic. In this project, it can identify a project life cycle (PLC) with five different phases. Those are identification, plan, implementation, testing and complete. As it is planned, new technologies are welcome during any phase. So this eventually becomes highly flexible. Even new ideas could arise while doing implementation. To create more competitive product, it should be up to date. This project basically can be defined as identification of an opportunity in the market. To meet up with these requirements, it is preferred to use an evolutionary/Iterative model as software development model and at the same time team would make sure to adhere with coding paradigms and standards.

1.2 Video Capturing & Analysis

After the research component conduct, it was able to found that there are several possible methods which can be used to achieve this objective but most of them are done using single cameras. In our case we may need to use multiple web cameras in order to capture the 3D motions of the user. Therefore we have to synchronize all those video inputs we capture using those web cameras. Then from that video we take snapshots for analysis user’s gestures. In this case we have to convert 3D in to 2D and then again into 3D, for do this operation we should use effective algorithms for reduce the data loss.

1.3 Image Mapping and Gesture Recognition

Image mapping and gesture recognition, together become an important part of the advanced human computer interaction systems. After capturing the images at a particular rate, as explained in 1.2 its next challenge to analyze them and recognize the 3D gesture of the object that is in front of the camera(s). Here is some research work on this area.

1.3.1 Euclidean Distance Fields

Euclidean Distant field technology helps in estimation of 3D object using a 2D image efficiently. An object’s distance field represents, for any point in space, the distance from that point to the object. The distance can be signed to distinguish between the inside and outside of an object. Estimating Euclidean distance from 2D range images is measured with Projected Distances from 2D image to the camera. Then it provides a 2D grid of line-of-sight distances from the camera to the object.

1.3.2 2D and 3D Analysis for Automated Gesture Recognition

The paper was published for the objective of creating a system that will be helping deaf people communication. It takes three data sets from cameras and analyzes them. The paper explains the technology as set by set. The first data set consists of images acquired with a single camera in a controlled light environment and a segmentation (called “2D segmentation”) based on luminance information. The second data set is acquired with a 3D camera which can produce a depth map; a segmentation (called “3D segmentation”) of the hand configurations based on the video and the depth map is performed. The third data set consists in 3D-segmented masks where the resulting hand mask is warped to compensate for hand pose variations.

1.4 Provide Scenery

In this project we are hoping to deliver a system which would enable interaction without any specific hardware development, but only with aid of software. We need to provide appropriate scene as required by the user. The user will enable to select one of following activities.
• Swimming in an aquarium/sea/river.
• Walking alone by seaside/riverside.
• Walking at a museum
• Drive a racing car
• Walking on moon/space
• Having a walk in a volcanic site
• Tsunami
• Surfing
• Snow board
We can add any kind of scenery since we are hoping to capture the whole body movement. We are hoping to design the graphical environment similar to natural conditions to some extend

1.5 Synchronization of Model and Scenery

According to the user movements the system will capture them. And then the environment should change according to the movement of the user. To estimate the motion we consider overall body movement. And respect to the motion it uses a neural network related thing and try to identify the motion. It will identify as the movement as related to the natural human movements. Then it would consider the real condition and the nature of the current environment. In the natural case, area of the scene changes only when eyes move. Other than that it should be the same angle of the environment. But objects of that area could affect by the user movement.
For example, consider the incident where the user is sitting alone by river side. Then environment should change but area remains the same. When user move head around the viewing angle/area changes. Now user picks a small piece of stone and throws it to the river. If user moves his hand in a vertical manner then it should sink at once. But if he moves hand in a horizontal plane then that piece of stone should jump about two to four times and sink.

Introduction

At a time where there are rapid advances in the field of Human Computer Interaction we have decided to give it another interpretation through the fulfillment of our project. This is one of the major emergent fields in the world of computer technology and people are all the way investigating to come up with advanced interactive interfaces. As it is mentioned above; the project “BubbleBoy" is an innovative, interactive 3D interface using full body motion. Obviously the interaction between the human and computers is done at the user interface which includes both software and hardware.

Sometimes one may think is this something to be invented as there are some applications already of same nature and in that case is there a room for a research component. Yes we also agree with the fact that there are applications with similar behavior as that of ours. But at the same time it should be noted that what we are expecting to do is not same as existing applications.

Nowadays we see lots of advanced user interactive interfaces are coming up to the market. Actually we can find that they are really eye catching and enriched with lot of features once we engage with them. But there are considerable amount of minus points when it is come to communication between computer and human, complexity of hardware and at the same time the software with prevalent systems. The concept of usability of the product is admired most and our main attention in developing the interfaces will be primarily based on that concept. A nice interface does not always mean a user friendly design. It is required badly an interface to be attractive. But at the same time it should fit with the ultimate users expectations. Otherwise the whole purpose of the introducing this product to the customers is lost if the users are not satisfied with the procedures they have to follow in order to facilitate out of this system and they anymore do not want the purchase the product.

That is the pathetic fate of the not user friendly yet unfortunately highly sophisticated products. The best example would be the some current implementations of ATM machines. At the time when the transaction is done the user is frustrated with the whole lot of questionnaire which has come across through the machine. And it is really a bad design of interfaces and a major drawback with most of that system. Likewise there are lots of systems in which they lack the capability of getting things much easier even for an average user and it should be simple enough to a novel person to be adapted with few steps. With contrast to a bad designs if we look into the reason behind the large popularity earned by the mobile phone it is designed in such a manner once the user dial the number of the person whom he or she wants to contact making a call is just a matter of pressing one button without undertaking so much steps.

By taken those facts into consideration in the scope of our project we will make sure that we design the interfaces always pertaining to the fact of ease of use. The interactive techniques we are going to exploit will not consume lot of steps to get to the business. And most importantly system will be configured in a way that has default settings for every stage that user may come across. After exploiting all these features it will be still capable of giving the same experience or even better than that when compared to the applications which use sophisticated hardware devices. For an instance think of a virtual environment which simulate an inside view of a swimming pool. The virtual environment is being subjected to change not only by the way user moves his hand but also by the speed at which user moves his hand. So in order to meet these expectations sound research components will be needed to choose the most optimal technology from rest of the optimal solutions that are available there and we have to come up with our own algorithms undoubtedly and it will realize the objectives of the course module as it is expected.

Moving onto the application areas we can see this concept is exploited as an effective remedial measure in various kinds of fields. Those includes environments for old and disabled people to have experiences which are difficult to have in real, movie industry for animated and real-life motion pictures, the video game industry to create assets for video games the science sector to create highly detailed models of chemical compounds, e-learning systems like virtual tutors to present learning materials, e-commerce systems to have electronic institutions to grab customers, e-tourism to foster tourism where tourists are impersonated as an avatar as well as host of uses depending on the application domain.

The BubbleBoy - voyage through virtual world...

This is a final year course module (CS 4200) project idea planned to do under Computer science and Engineering department of University of Moratuwa

This is a such an innovative framework for applications to give user to real time virtual experience. Using Multi-Move user interface and Using Web cameras detect whole body motions of Player and move the surrounding environment according to those actions. Also we suppose to use cheap components like Web cameras and not to use any tools or devices by user.

Team Name: Yathra

Group Members:

Rajini Dileepa

Hasitha Dilum Gunarathna

Thusitha Kumara
Lahiru Sandaruwan 

.

Project meeting minute - 03

Date & time, duration

The meeting was held on 2010- 10 -14 for about 30 minutes.

Venue

Office room of Dr. Chandana Gamage

Members present

All the members were present there.

Topic of the discussion

design decisions of several development aspects of the project and Requirements of the project

Brief Discussion of the discussion

In all our sessions with Chandana sir he emphasized the fact that the project we are doing is not just another programming project. Actually it is like a great chance we have given to try out the theoretical aspects of practical scenarios where Engineering concepts are governed. Since this a project carried out by final year under graduate students the standard of the project should be maintained at a certain level. In setting that standard certain aspects like Software Engineering, Engineering design and Engineering Economics have to take into consideration. And at the end of the day we should develop ourselves to a position where we can make design decisions by our own in terms of Engineering. As for the things need to carry out during next week Chandana sir ask us to come up with at least with five approaches to accomplish the task of 2D mapping to 3D. What he said was in selecting those approaches also they should be the best solutions which have been widely used, advanced and efficient. Along with those implementations we can just do test implementations and check whether they behave the way as it is expected. And other research area is to how the environment will react on the changes of the user. And another vital point made by the Chandana sir is to think about the end application at this stage of the development process itself. Although how much effort we have put in our project if the superficial appearance is not that appealing that would not attract the audience and our effort we have put in will not be not appreciated. So in a null shell the visual aspect is something which we can’t avoid and it should be treated with much care.  

Key topics discussed

1. Requirements of the project

2. things to carry out in next week

3. the state of end application

Comments and agreements

This week we agreed to use marker based system due the additional processing needed and the number of cameras that will be needed. What we decided was it should be a maximum of three preferably two.Finally Chandana sir assign each one with some research work for the week.

Project meeting minute - 02

Date & time, duration

The meeting was held on 2010- 10 -07.

Venue

Office room of Dr. Chandana Gamage

Members present

All the members were present there.

Topic of the discussion

Feedback for the draft document

Brief Discussion of the discussion

This contains the summery of the two meetings we have this during this week.Earlier we had meeting on 05th of this month. As for the day we were supposed to send a draft document of a project proposal. And at the time went there Dr. Chandana Gamage reviewed those documents and gave a detailed feedback. If I am to list down the comments made by him Improve on the ability of documentation related tasks · The theme of the project should change from “Advanced” to · “Innovative” What is expected from introduction section of the project proposal. · The points he mentioned were What are the dominant features of our system with compared to other § existing systems What are the qualities of bad designs and instances of them § The target audience of our project is not a set of people who are § highly technical Usability is always admired § We must stick into the fact that how we are going to provide § life-like experience using a computer to the target audience? We have to discuss what are the current implementations with same § behavior and what are the failures and limitations of them The term full body motion is really matters as it conveys the how the § environment is perceived by the user according to his actions and gestures. And another important comment he made was we have to follow some software process model so that it follows a systematic way of developing software product. And according to that we have to have timeline schedule and development phase in turn it would be useful to us also to get to know the health of the project and the current position where we are in it.

Key topics discussed

• Feedback for the draft Project proposal

• References of research papers

• technologies to be used for future developments

• Software process model

Comments and agreements

• Finalize on some software process mode

• not to include too much of technical things to the project proposal at this stage of the project.

• If we select some technology the rationale behind that implementation should be explained clearly

• research component should be carried out in order to find currently existing set of best schemes and what is the best solution out of them considering on pros and cons of each one of them

• put the year of the publication and give a brief introduction as to what it contains in a nut shell when mentioning about a reference

• to keep hard copies of important research papers with us.

• Add revision history to the document

• Improve our skill on writing

• Work on project proposal document according to feedback given

Project meeting minute - 01

Date & time, duration

The first meeting was held on 2010- 10 -01 for about 30 minutes.

Venue

Office room of Dr. Chandana Gamage

Members present

All the members were present there.

Topic of the discussion

Regarding writing the proposal

Brief Discussion of the discussion

Today we basically discuss the topic of what should be the content that should be included in the project proposal. The basic things were listed below.

 • Introduction

 • People involved

 • Theme detail

• This is advanced interactive interface to using full body motion

• Specifying of our system in which it will not have sophisticated hardware but the software

• The complexity will be mostly handled by the software

• Using commodity hardware

• Background research part

• Propose methodology to follow

• Deliverable

• Prototype system with prototype application

• Project schedule

Key topics discussed

1. 4D interactivity (what and how advanced interactive systems are built?)

2. how to combine interface with application framework

3. Application framework (provide scenery and provide motion to scenery changes mapping )

Comments and agreements

Chandana sir asked us to come up with draft project proposal in the next meeting and do the researches on key topic areas he has already mentioned.