Kinect development of Interaction interaction design

Recommended for you: Get network issues from WhatsUp Gold. Not end users.

Making Kinect 1.7 and later versions of the SDK, the biggest change is the addition of the Kinect Interaction and Kinect Fusion. Kinect Interactions provides some new with a gesture recognition controls such as the push-to-press button, the grip-to-pan list control, and support for multiple users at the same time, the interaction of two people, the new addition to the control can be integrated into the application is very convenient, greatly simplify the development and debugging process.

Making in Kinect Developer ToolKit, Kinect Interactions provides, 1920*1080 need to run the Kinect Interactions resolution. To provide this blog briefly show Kinect Interactions in the new controls and interactive mode.

 

A,  To establish the necessary environment

Before create project, need to go to the official website to download and install the Kinect SDK V1.7 and Developer Toolkit. First, open the Visual Studio to create a simple WPF desktop application, then add Microsoft.Kinect.dll, Kinect.Toolkit.dll, Kinect.Toolkit.Controls.dll and Kinect.Toolkit.Interaction.dll references, these DLL general in the installation directory, my computer is on the path

C:\Program Files\Microsoft SDKs\Kinect\v1.7\Assemblies\

And C:\Program Files\Microsoft SDKs\Kinect\Developer Toolkit v1.7.0\Assemblies\

 

1.1 using KinectSensorChooserControls the initialization of KinectSensor

Making in the Microsoft.Kinect.Toolkit.Controls namespace, first used in the control could be KinectSensorChooserUI, it is used to indicate the current working state of Kinect, prompting the user to the Kinect sensor is working properly, for example, is broken, is inserted into the USB interface on wrong.

Making you want to add this control, first in the main form, add the following namespace:

 

xmlns:k="http://schemas.microsoft.com/kinect/2013"
 

Making then, add the KinectSensorChooserUI control in the main form, the code as follows:

<Grid>
    <k:KinectSensorChooserUI HorizontalAlignment="Center" VerticalAlignment="Top" Name="sensorChooserUI" />
</Grid>
 

 

In CS code, we need to initialize the KinectSensorChooser object:

/// <summary>
/// Create a KinectSensorChooserUI object, to indicate the current working state of Kinect, 
/// Prompts the user to the Kinect sensor is working properly, for example, is broken, is inserted into the USB interface on wrong.
/// </summary>
private KinectSensorChooser sensorChooser;
 

 

Then register the OnLoad event in the main form of the constructor

/// <summary>
/// The main function of the form
/// </summary>
public MainWindow()
{
    InitializeComponent();
    //Load initialization method
    Loaded += OnLoaded;
}
 

 

Commissioned by the method of creating OnLoaded:  

 

/// <summary>
/// Method of loading the form runs
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void OnLoaded(object sender, RoutedEventArgs e)
{
    this.sensorChooser = new KinectSensorChooser();
    this.sensorChooser.KinectChanged += SensorChooserOnKinectChanged;
    this.sensorChooserUi.KinectSensorChooser = this.sensorChooser;
    this.sensorChooser.Start(); 
}
  

 

Making if the sensor changes state, such as closing or initialization is complete, it will trigger the SensorChooserOnKinectChanged event. Now in order to demonstrate the convenience, we use MessageBox pop-up prompt information:  

 

private void SensorChooserOnKinectChanged(object sender, KinectChangedEventArgs args)
{
    MessageBox.Show(args.NewSensor == null ? "No Kinect" : args.NewSensor.Status.ToString()); 
}
 

 

Making power plug, KinectSensorChooserUI will switch to the initialization mode, move the mouse to start, to prompt initializing:

 

5 Kinect initializing tips

   

Making and wait for a while, after the initialization is done, you can see the Kinect icon, and then state to the initialization is successful, then the callback method will be implemented, the pop-up dialog box MessageBox. After the initialization, the icon will disappear from the screen, until the Kinect state change again:

6 Kinect connected 

 

Making KinectSensorChooserUI control according to different state will try to give the user the corresponding prompt, switch to USB port, power line inspection of Kinect is even better.

 

1.2 building KinectBasic environmental interaction

Making now we already connected to the Kinect, the basic settings to start making some interaction. Now the SensorChooserOnKinectChanged event code completion:

 

/// <summary>
/// The sensor state changed event
/// </summary>
/// <param name="sender"></param>
/// <param name="args"></param>
private void SensorChooserOnKinectChanged(object sender, KinectChangedEventArgs args)
{
    //Kinect sensor processing initial off, even on the new sensor and open the depth and skeletal data stream.
    bool error = false;//Whether the Kinect sensor initial
    if (args.OldSensor != null)
    {
        try
        {
            args.OldSensor.DepthStream.Range = DepthRange.Default;
            args.OldSensor.SkeletonStream.EnableTrackingInNearRange = false;
            args.OldSensor.DepthStream.Disable();
            args.OldSensor.SkeletonStream.Disable();
        }
        catch (InvalidOperationException)
        {
            // KinectSensor might enter an invalid state while enabling/disabling streams or stream features.
            // E.g.: sensor might be abruptly unplugged.
            error = true;
        }
    }

    if (args.NewSensor != null)
    {
        try
        {
            //Open the sensor depth data flow
            args.NewSensor.DepthStream.Enable(DepthImageFormat.Resolution640x480Fps30);
            //Open the skeletal data stream
            args.NewSensor.SkeletonStream.Enable();

            try
            {
                //Use the default close mode, K4W can use close mode, Xbox does not support the close model
                args.NewSensor.DepthStream.Range = DepthRange.Near;
                args.NewSensor.SkeletonStream.EnableTrackingInNearRange = true;
                args.NewSensor.SkeletonStream.TrackingMode = SkeletonTrackingMode.Seated;
            }
            catch (InvalidOperationException)
            {
                // Non Kinect for Windows devices do not support Near mode, so reset back to default mode.
                args.NewSensor.DepthStream.Range = DepthRange.Default;
                args.NewSensor.SkeletonStream.EnableTrackingInNearRange = false;
            }
        }
        catch (InvalidOperationException)
        {
            // KinectSensor might enter an invalid state while enabling/disabling streams or stream features.
            // E.g.: sensor might be abruptly unplugged.
            error = true;
           
        }
        try{
            if(!error)
            {
                //The KinectSensor property is set to kinectRegion to get kinectSensor.
                kinectRegion.KinectSensor = args.NewSensor; 
            }
        }
        catch(Exception)
        {
            throw;
        }
    }
}

 

 

Making the above code can handle Kinect sensor old pulled out, even on the new sensor and open the depth and skeletal data stream scenario, in the development, we may be connected to a plurality of sensors at the same time. In the try code, we try to open close mode, if the Xbox sensor by the user, not support, then go to the general mode. In general, open close model debugging and convenient user use of close.

 

Two, KinectInteractive control

2.1 add Kinect Region Control

Making to want to use Kinect Interaction, KinectRegion need to add a container in the interface. KinectRegion is a key element in WPF using the Kinect Interaction interaction. It is a container for other Kinect interaction control. The KinectRegion is also responsible for display and mobile hand icon, namely user's hand skeleton points at the interface of the show. Applications on the main interface can have multiple KinectRegion, but each KinectRegion cannot be nested, each KinectRegion can have its own Kinect sensor objects.

 

Making now, add a KinectRegion object to the main form.:

 

<k:KinectRegion Name="kinectRegion"></k:KinectRegion>
   

 

Making then, set the KinectSensor property to kinectRegion to our acquisition of kinectSensor:

 

if (!error)
    kinectRegion.KinectSensor = args.NewSensor;
  

 

Making if you run the program, it will prompt error:

7 cannot find error

   

Making this mistake is very common, suggesting that KinectInteraction170_32.dll not found. Where to find? The easiest way is to install path following Kinect SDK and Toolkit to search, find, according to the current operating system version, will copy the 32 bit or 64 bit DLL to the output path project. Recommend the use of softwareeverythingLook, a software only 400K, search for files of ultrafast.



  

 

Making run the application again, may need to initialize the 10-20s event identification, if everything is normal, it will display the hand shaped cursor on the application:

13 Kinect Handcursor

Making for screenshots and demonstration, here is a little skill, due to the demonstration, demonstration time must be a certain distance Kinect distance, so that the hand can't reach the PrintScreen key on the keyboard, so we recommend the use of Kinect Studio to record Kinect data, and then broadcast the data to the debugging of Kinect application program, such can greatly improved the efficiency of debugging, don't always have to stand in front of Kinect operation. I put the action recorded, then the screenshots or record screen animation, so a lot of convenience.

 

Add a KinectUserViewer control 2.2

Making sometimes in the application program, we hope to be able to display the user's own image, so as to give some feedback, KinectUserViewer control in Kinect.Toolkit.Controls, can use the control to display the depth image characters, to use it, you need to define a KinectUserViewer object on the main interface.:

 

<k:KinectUserViewer VerticalAlignment="Top" HorizontalAlignment="Center"  k:KinectRegion.KinectRegion="{Binding ElementName=kinectRegion}" Height="100" />
  

Making KinectUserViewer shows the characters of depth image data in Kinect, if you can't see the image, so that the current Kinect does not track to you, at this time should try to let the Kinect to track your mobile. Run the program, or just Kinect Studio recorded data, then you can see for yourself in the Kinect people in the field of view of the depth image.

9 KinectUserView Control

Before making controls used to show the current state of Kinect, hand positions, and the characters of depth image in Kinect, feedback which can give the user some global, said the current system runs normally, now we have to use some of the basic control to real and applications to interact.

 

 

2.3 add KinectTileButton picture button

Making is the first to introduce the KinectTileButton control, a control, he is the most simple, a bit like the Win8 inside the Metro patch, he is actually a normal button control, but the Kinect by hand to simulate mouse behavior.

<k:KinectTileButton Label="Press me!" Click="ButtonOnClick"></k:KinectTileButton>
 

Making then write some hints in the click event.:

/// <summary>
/// The KinectTileButton button to listen for events
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void ButtonOnClick(object sender, RoutedEventArgs e)
{
    //Press the button pop-up hints are given
    MessageBox.Show("You clicked me!!!");
}

Making when we hand does not touch the controls, he default is blue. When the hand is moved to the button, the button will become large, and the hand icon will appear around the shadow, to display the current in a suspended state. Then we will push hands like Kinect, will find the hand icon will according to the distance to Kinect distance to generate animation, that we are "pressure" button. When they reach a certain threshold depth, said users have to press the button, the hand icon will change, and will trigger the Click event.

10 KinectTileButtonControl Small

 

2.4 add KinectCircleButton picture button

Making KinectCircleButton and KinectTileButton are similar, but the button shape is round, this can be seen from the name. To add KinectCircleButton to the interface layout in the existing, we need to slightly adjust. KinectRegion is a content controls, said there is only a child control, however, the child control can be a contain controls other containers such as Grid or StackPanel. We need the prior to the KinectTileButton contains a Grid control now, then KinectTileButton is moved to the left, and then add a KinectCircleButton control to the right.

 

<k:KinectCircleButton Label="Circle" HorizontalAlignment="Right" Height="200" VerticalAlignment="Top" Click="ButtonOnClick" >Hi</k:KinectCircleButton>
 

Making repeat, can see a similar animation effects:

11 KinectCircleButton Control Small

Making needs to be pointed out is, the two controls can also operate with the mouse.

 

Add a KinectScrollViewer Control 2.5

Making before Kinect 1.7 SDK, to realize the use of Kinect to navigate through the list, and select is more difficult, but the introduction of the new 1.7 SDK controls KinectScrollViewer greatly simplifies the operation. Now, we put a KinectScrollViewer in the bottom of the form. Add the following code to the KinectRegion internal Grid:

<k:KinectScrollViewer VerticalScrollBarVisibility="Disabled" HorizontalScrollBarVisibility="Auto" VerticalAlignment="Bottom">
    <StackPanel Orientation="Horizontal" Name="scrollContent" />
</k:KinectScrollViewer>

Making this control as standard in WPF ScrollViewer. There is a StackPanell used for the placement of the rolling elements, the two controls can makes the content level of rolling, put some KinectCircleButtons. in StackPanel now

/// <summary>
/// Method of loading the form runs
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void OnLoaded(object sender, RoutedEventArgs e)
{
    this.sensorChooser = new KinectSensorChooser();
    this.sensorChooser.KinectChanged += SensorChooserOnKinectChanged;
    this.sensorChooserUi.KinectSensorChooser = this.sensorChooser;
    this.sensorChooser.Start();
     

    //Fill in scrollView.
    for (int i = 1; i <20; i++)
    {
        //Kinect Interaction round button
        var button = new KinectCircleButton
        {
            Content = i,
            Height = 200
        };

        int i1 = i;
        //For the button to register event, press gives pop-up box
        button.Click +=
            (o, args) => MessageBox.Show("You clicked button #" + i1);

        scrollContent.Children.Add(button);
    }
}

 

Making now run the program, As shown in figure animation, At the beginning of the, You can see the following KinectCircleButton; when the hand moving hand to scroll area, Color change; and then close the fingers, The fist, Gesture icon will change, You are currently on the KinectScrollViewer fetch operation, Mobile fist, The KinectScrollViewer control the icon of a hand attached to the control, As our fingers on the touch screen operation., Then the button will follow the gesture and move, After a good location, Unclench your fist, Stop moving, This time, Towards the desired button, The hand push forward, Just like in front of the controls that trigger the Click event.

 12 KinectScrollViewer Control  small

  

Making because I install the msysgit tools, then do not know GitHub graphics tools also inexplicable installed, plus account still another computer use, leading to public key added has been invalid, had go through untold hardships finally settled, remind everyone, is best not to use the official graphic tools, anyway I feel uncomfortable; Google msysgit tool is good, but also recommend git bash command line tools it.

 

Making gibberish did not say more, speak frankly and sincerely to GitHub code: https://github.com/Marsyangkang/KinectInteraction

 

Kinect series of Bo Wen:  

Kinect development of Powerpoint assistant

 
Interaction design Kinect development Interaction

 
Kinect development with Unity3D game application development

 
Development of Kinect somatosensory raised his hand to program design (Kinect Tetris)

 
Kinect development of simple gesture recognition

 
Kinect development of skeletal joint data acquisition and drawn into the match

 
Kinect development of the acquisition data color camera
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download

Posted by Lulu at December 02, 2013 - 12:47 PM