Google cardboard, color-cross

View previous topic View next topic Go down

Google cardboard, color-cross

Post by kenran on Thu Feb 12, 2015 9:46 am

Hi everybody,

I tried google cardboard with my smartphone and though it would be awesome to use in xnalara.

Even if you don't have a gyro in your phone (only premium), no pb you can use a better solution head tracking using a webcam and trackir or freetrack( any vids on youtube)


I stream xnalara window content using Kainy server and Kainy with ads android client.
Now i would like to make a plugin or add a functionality in xnalara xps to make a stereoscopic view.

Anybody tried to develop it ?
Where can i find xps source code ? is there a repository ?

thank you

kenran
XNALara guest
XNALara guest

Male Сообщения : 7

Back to top Go down

Re: Google cardboard, color-cross

Post by KaRaT3KiD on Mon Feb 16, 2015 5:50 pm

Interesting idea, but as I know, the source code for XPS is unavailable, but the code for XNALara is available at SourceForge (please use Google, I can't post links in the first 7 days).

KaRaT3KiD
XNALara guest
XNALara guest

Male Сообщения : 1
Age : 17
Location : Hungary

Back to top Go down

Re: Google cardboard, color-cross

Post by XNAaraL on Mon Feb 16, 2015 6:45 pm

KaRaT3KiD wrote:Interesting idea, but as I know, the source code for XPS is unavailable, but the code for XNALara is available at SourceForge (please use Google, I can't post links in the first 7 days).

^^^ Right, google is your friend ... and this forum also Wink
Laughing
I am one of the XNALara/XPS developer ... and I know this.
The link is here [You must be registered and logged in to see this link.]
Code:
http://xnalara.home-forum.com/t167-source-code
The SVN is there [You must be registered and logged in to see this link.]
Code:
http://sourceforge.net/p/xnalara/code/HEAD/tree/XNALara/Renderer.cs

About the new Stereoscopy ( [You must be registered and logged in to see this link.] ) 3D feature  ... it is as "easter egg"  ( [You must be registered and logged in to see this link.] )  in XPS since long time Wink

[You must be registered and logged in to see this link.]
Code:
http://www.tombraiderforums.com/showpost.php?p=6267302&postcount=79

BTW:
Since XPS 10.9.8.7.6.5.4 --  Countdown Script Edition  -- release 172 (http://xnalara.home-forum.com/t269-xna-posing-studio-10-9-8-7-6-5-4-countdown-script-edition) ... XPS have a "script interface" for writing extensions (plugins).
However, I recommend to implement the extension using a DrawableGameComponent (Like the Bloom effect, the "Display Normals" effect .... or the easter eggs "Mini game" and "Stereoscopy") [You must be registered and logged in to see this link.]
Code:
http://mort8088.com/2011/02/28/xna-fps-drawable-game-component/

Code:

    /// <summary>
    /// This is a game component that implements IUpdateable.
    /// </summary>
    public class RenderStereoscopyComponent : Microsoft.Xna.Framework.DrawableGameComponent {

        private Game game;
        // ...

        public RenderStereoscopyComponent(Game game)
            : base(game)
        {
            this.game = game;
            DrawOrder = 10;
            game.Components.Add(this);
        }

        /// <summary>
        /// Allows the game component to perform any initialization it needs to before starting
        /// to run.  This is where it can query for any required services and load content.
        /// </summary>
        public override void Initialize()
        {
            // TODO: Add your initialization code here
            base.Initialize();
        }

        /// <summary>
        /// Allows the game component to update itself.
        /// </summary>
        /// <param name="gameTime">Provides a snapshot of timing values.</param>
        public override void Update(GameTime gameTime)
        {
            // TODO: Add your update code here
            base.Update(gameTime);
        }

       #region Draw
        /// <summary>
        /// This is where it all happens. Grabs a scene that has already been rendered,
        /// and uses postprocess magic to add the same view with a different camera angle.
        /// </summary>
        public override void Draw(GameTime gameTime)
        {
            InitRenderTargets(format, game.Window.ClientBounds.Width, game.Window.ClientBounds.Height);
            // Resolve the scene into a texture, so we can
            // use it as input data for the stereoscopy.
            GraphicsDevice.ResolveBackBuffer(resolveTarget);
          ....
        }
       #endregion Draw

I am sure, you know what to do ... and your color-cross is quickly done. Good luck and show us your result.


XNAaraL
XNALara teacher
XNALara teacher

Male Сообщения : 688
Location : Германия

Back to top Go down

Re: Google cardboard, color-cross

Post by kenran on Tue Feb 17, 2015 8:11 am

Wow thank you everybody.
Yes i found xnalara code. I tried to open it with visual c# 2003,2005,2010 without success. I will try today with 2008.
It's very easy to code stereoscopy, it's just a 65mm translation (distance between the 2 eyes).
Plugin extension is the best solution indeed.
I will post maybe this weekend a first beta plugin and a complete tuto.

kenran
XNALara guest
XNALara guest

Male Сообщения : 7

Back to top Go down

Re: Google cardboard, color-cross

Post by XNAaraL on Tue Feb 17, 2015 9:40 am

kenran wrote:Wow thank you everybody.
Yes i found xnalara code. I tried to open it with visual c# 2003,2005,2010 without success. I will try today with 2008.
It's very easy to code stereoscopy, it's just a 65mm translation (distance between the 2 eyes).
Plugin extension is the best solution indeed.
I will post maybe this weekend a first beta plugin and a complete tuto.

VC# 2008 is the right one.
[You must be registered and logged in to see this link.]
Code:
http://comments.deviantart.com/1/427330899/3343862166
Spoiler:

The VC# "Version" is

- Visual Studio 2008 (Express)

- XNA Game Studio v3.0

- XNA Framework 3.0

- .NET Framework 3.5

XNAaraL
XNALara teacher
XNALara teacher

Male Сообщения : 688
Location : Германия

Back to top Go down

Re: Google cardboard, color-cross

Post by kenran on Thu Feb 19, 2015 5:41 pm

OK, i'm back

This is my first script attempt :


Code:
using System;
using System.Collections.Generic;
using System.Linq;
using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Audio;
using Microsoft.Xna.Framework.Content;
using Microsoft.Xna.Framework.GamerServices;
using Microsoft.Xna.Framework.Graphics;
using Microsoft.Xna.Framework.Input;
using Microsoft.Xna.Framework.Media;
using Microsoft.Xna.Framework.Net;
using Microsoft.Xna.Framework.Storage;


namespace XNALara
{
    /// <summary>
    /// This is a game component that implements IUpdateable.
    /// </summary>
    public class StereoVR : Microsoft.Xna.Framework.DrawableGameComponent
    {

        private Game game;
        ResolveTexture2D renderTargetLeft;
        ResolveTexture2D renderTargetRight;
        SpriteBatch spriteBatch;
        int renderTargetWidth, renderTargetHeight;
        Rectangle leftRectangle;
        Rectangle rightRectangle;
        Boolean initialized;

        public StereoVR(Game game)
            : base(game)
        {
            // TODO: Construct any child components here
            this.game = game;
            DrawOrder = 10;
            initialized = false;
            game.Window.ClientSizeChanged += new EventHandler(WindowSizeChanged);

            game.Components.Add(this);
        }

        private void WindowSizeChanged(object sender, EventArgs e)
        {
            initialized = false;
        }

        /// <summary>
        /// Allows the game component to perform any initialization it needs to before starting
        /// to run.  This is where it can query for any required services and load content.
        /// </summary>
        public override void Initialize()
        {
            // TODO: Add your initialization code here
            base.Initialize();
        }

        /// <summary>
        /// Allows the game component to update itself.
        /// </summary>
        /// <param name="gameTime">Provides a snapshot of timing values.</param>
        public override void Update(GameTime gameTime)
        {
            // TODO: Add your update code here
            base.Update(gameTime);
        }

        Vector3 campos;

        #region Draw
        /// <summary>
        /// This is where it all happens. Grabs a scene that has already been rendered,
        /// and uses postprocess magic to add the same view with a different camera angle.
        /// </summary>
        public override void Draw(GameTime gameTime)
        {
            //System.Diagnostics.Debug.WriteLine("stereo draw");

            if (!initialized)
            {
                spriteBatch = new SpriteBatch(GraphicsDevice);
                renderTargetWidth = game.Window.ClientBounds.Width;
                renderTargetHeight = game.Window.ClientBounds.Height;
                leftRectangle = new Rectangle(0, 0, renderTargetWidth / 2, renderTargetHeight);
                rightRectangle = new Rectangle(renderTargetWidth / 2, 0, renderTargetWidth / 2, renderTargetHeight);
                InitRenderTargets();
                initialized = true;
            }

            
            // Resolve the scene into a texture, so we can
            // use it as input data for the stereoscopy.
            GraphicsDevice.ResolveBackBuffer(renderTargetLeft);
            
            // change camera view (translation)
            campos = game.Camera.Target;
            campos.X = campos.X + (float)(0.1);
            game.Camera.Target = campos;

            // render
            // -----------------

            game.GraphicsDevice.Clear(ClearOptions.Target | ClearOptions.DepthBuffer, game.BackgroundColor, 1.0f, 0);
            if (game.BackgroundImage != null)
            {
                game.Renderer.RenderBackgroundImage(0, 0, 0, 0, game.BackgroundImage.ScaleX, game.BackgroundImage.ScaleY);
            }

            game.Renderer.RenderSceneFull(false);

            if (game.IsPostProcessingActive)
            {
                game.Renderer.ApplyPostProcessing();
            }

            if (game.DisplayBones)
            {
                game.BoneSelector.Render();
            }

            //hud.Render();

            //-----------

            GraphicsDevice.ResolveBackBuffer(renderTargetRight);

            campos.X = campos.X - (float)(0.1);
            game.Camera.Target = campos;

            spriteBatch.Begin();
            spriteBatch.Draw(renderTargetLeft, leftRectangle, Color.White);
            spriteBatch.Draw(renderTargetRight, rightRectangle, Color.White);
            spriteBatch.End();
            

            base.Draw(gameTime);
            
        }
        #endregion Draw


        private void InitRenderTargets()
        {
           // System.Diagnostics.Debug.WriteLine("width:"+renderTargetWidth);
           // System.Diagnostics.Debug.WriteLine("Height:"+renderTargetHeight);
            renderTargetLeft = new ResolveTexture2D(GraphicsDevice,
                renderTargetWidth, renderTargetHeight, 1,
                GraphicsDevice.PresentationParameters.BackBufferFormat);

            renderTargetRight = new ResolveTexture2D(GraphicsDevice,
                renderTargetWidth, renderTargetHeight, 1,
                GraphicsDevice.PresentationParameters.BackBufferFormat);
        }
    }
}

in game.cs i register my component :

Code:
protected override void Initialize() {
            graphicsDeviceManager.PreferredBackBufferWidth = DefaultCanvasWidth;
            graphicsDeviceManager.PreferredBackBufferHeight = DefaultCanvasHeight;
            graphicsDeviceManager.IsFullScreen = false;
            graphicsDeviceManager.ApplyChanges();

            this.IsFixedTimeStep = false;
            this.IsMouseVisible = true;

            StereoVR svr = new StereoVR(this);
          

            base.Initialize();
        }

Now i have 3 problems :
- Moving the target camera is not a translation. I need to reverse calculate the camera distance+target from position
- There is a lot of lag when i switch to other programs
- I cannot load my Dynasty warrior or DOA models  Sad  because the mesh format is unknown even with "xps export to old format", this code is too old Sad
If somebody have a working model with this version (9.7.7).

Here the result with dummy model :
Spoiler:

kenran
XNALara guest
XNALara guest

Male Сообщения : 7

Back to top Go down

Re: Google cardboard, color-cross

Post by XNAaraL on Fri Feb 20, 2015 7:02 am

kenran wrote:OK, i'm back

This is my first script attempt :

...

Now i have 3 problems :
- Moving the target camera is not a translation. I need to reverse calculate the camera distance+target from position
- There is a lot of lag when i switch to other programs
- I cannot load my Dynasty warrior or DOA models  Sad  because the mesh format is unknown even with "xps export to old format", this code is too old Sad
If somebody have a working model with this version (9.7.7).

Here the result with dummy model :
Spoiler:


  1. Right, moving the target camera is not a scene Translation. So I had spoken about camera angle.
  2. Two render needs more performance than one render ... of course. but this should not be cause a LAG.
  3. There is no XPS Option like "Export to old >non-generic format" ... confused.
  4. The old DoF Format required:

    1. The mesh Name must be one of this names [You must be registered and logged in to see this link.] ...
    2. the mesh must have the right amount of mesh parts ... with the mesh part names matching the used model loader from there [You must be registered and logged in to see this link.]
    3. the  the old tutorials are there [You must be registered and logged in to see this link.]


  5. The offical models are there [You must be registered and logged in to see this link.]
    Code:
    http://www.mediafire.com/?oan90n8923obi


XNAaraL
XNALara teacher
XNALara teacher

Male Сообщения : 688
Location : Германия

Back to top Go down

Re: Google cardboard, color-cross

Post by kenran on Fri Feb 20, 2015 2:14 pm

Code Update :

- Camera angle computed according the distance from camera to the target ( need test to adjust factor )
- pixel ratio is now 1:1
- lara model loaded but back culling pb even with BFC on and always

Code:
using System;
using System.Collections.Generic;
using System.Linq;
using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Audio;
using Microsoft.Xna.Framework.Content;
using Microsoft.Xna.Framework.GamerServices;
using Microsoft.Xna.Framework.Graphics;
using Microsoft.Xna.Framework.Input;
using Microsoft.Xna.Framework.Media;
using Microsoft.Xna.Framework.Net;
using Microsoft.Xna.Framework.Storage;


namespace XNALara
{
    /// <summary>
    /// This is a game component that implements IUpdateable.
    /// </summary>
    public class StereoVR : Microsoft.Xna.Framework.DrawableGameComponent
    {

        private Game game;
        ResolveTexture2D renderTargetLeft;
        ResolveTexture2D renderTargetRight;
        SpriteBatch spriteBatch;
        int renderTargetWidth, renderTargetHeight;
        Rectangle leftRectangle, rightRectangle, srcRectangle;
        Boolean initialized;

        public StereoVR(Game game)
            : base(game)
        {
            // TODO: Construct any child components here
            this.game = game;
            DrawOrder = 10;
            initialized = false;
            game.Window.ClientSizeChanged += new EventHandler(WindowSizeChanged);

            game.Components.Add(this);
        }

        private void WindowSizeChanged(object sender, EventArgs e)
        {
            initialized = false;
        }

        /// <summary>
        /// Allows the game component to perform any initialization it needs to before starting
        /// to run.  This is where it can query for any required services and load content.
        /// </summary>
        public override void Initialize()
        {
            // TODO: Add your initialization code here
            base.Initialize();
        }

        /// <summary>
        /// Allows the game component to update itself.
        /// </summary>
        /// <param name="gameTime">Provides a snapshot of timing values.</param>
        public override void Update(GameTime gameTime)
        {
            // TODO: Add your update code here
            base.Update(gameTime);
        }

        float camRotH;
        float camRotV;
        float angle;

        #region Draw
        /// <summary>
        /// This is where it all happens. Grabs a scene that has already been rendered,
        /// and uses postprocess magic to add the same view with a different camera angle.
        /// </summary>
        public override void Draw(GameTime gameTime)
        {

            if (!initialized)
            {
                spriteBatch = new SpriteBatch(GraphicsDevice);
                renderTargetWidth = game.Window.ClientBounds.Width;
                renderTargetHeight = game.Window.ClientBounds.Height;
                leftRectangle = new Rectangle(0, 0, renderTargetWidth / 2, renderTargetHeight);
                rightRectangle = new Rectangle(renderTargetWidth / 2, 0, renderTargetWidth / 2, renderTargetHeight);
                srcRectangle = new Rectangle(renderTargetWidth / 4, 0, renderTargetWidth / 2, renderTargetHeight);
                InitRenderTargets();
                initialized = true;
            }

            
            // Resolve the scene into a texture, so we can
            // use it as input data for the stereoscopy.
            GraphicsDevice.ResolveBackBuffer(renderTargetLeft);
            
            // angle is different if object is far or close
            // TAN angle =  (distance neck to eye)/2  /  camera.distance
            angle = (float)(Math.Atan2(0.0325f, game.Camera.Distance)) * 2f;
            camRotH = game.Camera.RotationHorizontal;
            camRotV = game.Camera.RotationVertical;
            camRotH = camRotH + angle;
            game.Camera.SetRotation(camRotH, camRotV);

            // render
            // -----------------

            game.GraphicsDevice.Clear(ClearOptions.Target | ClearOptions.DepthBuffer, game.BackgroundColor, 1.0f, 0);
            if (game.BackgroundImage != null)
            {
                game.Renderer.RenderBackgroundImage(0, 0, 0, 0, game.BackgroundImage.ScaleX, game.BackgroundImage.ScaleY);
            }

            game.Renderer.RenderSceneFull(false);

            if (game.IsPostProcessingActive)
            {
                game.Renderer.ApplyPostProcessing();
            }

            if (game.DisplayBones)
            {
                game.BoneSelector.Render();
            }

            ///hud.Render();

            //-----------

            GraphicsDevice.ResolveBackBuffer(renderTargetRight);

            // change back camera params
            camRotH = camRotH - angle;
            game.Camera.SetRotation(camRotH, camRotV);

            spriteBatch.Begin();
            spriteBatch.Draw(renderTargetLeft, leftRectangle, srcRectangle ,Color.White);
            spriteBatch.Draw(renderTargetRight, rightRectangle, srcRectangle, Color.White);
            spriteBatch.End();
            

            base.Draw(gameTime);
            
        }
        #endregion Draw


        private void InitRenderTargets()
        {
           // System.Diagnostics.Debug.WriteLine("width:"+renderTargetWidth);
           // System.Diagnostics.Debug.WriteLine("Height:"+renderTargetHeight);
            renderTargetLeft = new ResolveTexture2D(GraphicsDevice,
                renderTargetWidth, renderTargetHeight, 1,
                GraphicsDevice.PresentationParameters.BackBufferFormat);

            renderTargetRight = new ResolveTexture2D(GraphicsDevice,
                renderTargetWidth, renderTargetHeight, 1,
                GraphicsDevice.PresentationParameters.BackBufferFormat);
        }
    }
}

Preview :

Spoiler:

Now i'm working on head tracker client connection (maybe with telnet) and doom like keyboard shortcuts to move camera

Here a little tutorial :

[You must be registered and logged in to see this link.]

kenran
XNALara guest
XNALara guest

Male Сообщения : 7

Back to top Go down

Re: Google cardboard, color-cross

Post by XNAaraL on Fri Feb 20, 2015 6:13 pm

kenran wrote:Code Update :

- Camera angle computed according the distance from camera to the target ( need test to adjust factor )
- pixel ratio is now 1:1
- lara model loaded but back culling pb even with BFC on and always

Spoiler:

Code:
using System;
using System.Collections.Generic;
using System.Linq;
using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Audio;
using Microsoft.Xna.Framework.Content;
using Microsoft.Xna.Framework.GamerServices;
using Microsoft.Xna.Framework.Graphics;
using Microsoft.Xna.Framework.Input;
using Microsoft.Xna.Framework.Media;
using Microsoft.Xna.Framework.Net;
using Microsoft.Xna.Framework.Storage;


namespace XNALara
{
    /// <summary>
    /// This is a game component that implements IUpdateable.
    /// </summary>
    public class StereoVR : Microsoft.Xna.Framework.DrawableGameComponent
    {

        private Game game;
        ResolveTexture2D renderTargetLeft;
        ResolveTexture2D renderTargetRight;
        SpriteBatch spriteBatch;
        int renderTargetWidth, renderTargetHeight;
        Rectangle leftRectangle, rightRectangle, srcRectangle;
        Boolean initialized;

        public StereoVR(Game game)
            : base(game)
        {
            // TODO: Construct any child components here
            this.game = game;
            DrawOrder = 10;
            initialized = false;
            game.Window.ClientSizeChanged += new EventHandler(WindowSizeChanged);

            game.Components.Add(this);
        }

        private void WindowSizeChanged(object sender, EventArgs e)
        {
            initialized = false;
        }

        /// <summary>
        /// Allows the game component to perform any initialization it needs to before starting
        /// to run.  This is where it can query for any required services and load content.
        /// </summary>
        public override void Initialize()
        {
            // TODO: Add your initialization code here
            base.Initialize();
        }

        /// <summary>
        /// Allows the game component to update itself.
        /// </summary>
        /// <param name="gameTime">Provides a snapshot of timing values.</param>
        public override void Update(GameTime gameTime)
        {
            // TODO: Add your update code here
            base.Update(gameTime);
        }

        float camRotH;
        float camRotV;
        float angle;

        #region Draw
        /// <summary>
        /// This is where it all happens. Grabs a scene that has already been rendered,
        /// and uses postprocess magic to add the same view with a different camera angle.
        /// </summary>
        public override void Draw(GameTime gameTime)
        {

            if (!initialized)
            {
                spriteBatch = new SpriteBatch(GraphicsDevice);
                renderTargetWidth = game.Window.ClientBounds.Width;
                renderTargetHeight = game.Window.ClientBounds.Height;
                leftRectangle = new Rectangle(0, 0, renderTargetWidth / 2, renderTargetHeight);
                rightRectangle = new Rectangle(renderTargetWidth / 2, 0, renderTargetWidth / 2, renderTargetHeight);
                srcRectangle = new Rectangle(renderTargetWidth / 4, 0, renderTargetWidth / 2, renderTargetHeight);
                InitRenderTargets();
                initialized = true;
            }

            
            // Resolve the scene into a texture, so we can
            // use it as input data for the stereoscopy.
            GraphicsDevice.ResolveBackBuffer(renderTargetLeft);
            
            // angle is different if object is far or close
            // TAN angle =  (distance neck to eye)/2  /  camera.distance
            angle = (float)(Math.Atan2(0.0325f, game.Camera.Distance)) * 2f;
            camRotH = game.Camera.RotationHorizontal;
            camRotV = game.Camera.RotationVertical;
            camRotH = camRotH + angle;
            game.Camera.SetRotation(camRotH, camRotV);

            // render
            // -----------------

            game.GraphicsDevice.Clear(ClearOptions.Target | ClearOptions.DepthBuffer, game.BackgroundColor, 1.0f, 0);
            if (game.BackgroundImage != null)
            {
                game.Renderer.RenderBackgroundImage(0, 0, 0, 0, game.BackgroundImage.ScaleX, game.BackgroundImage.ScaleY);
            }

            game.Renderer.RenderSceneFull(false);

            if (game.IsPostProcessingActive)
            {
                game.Renderer.ApplyPostProcessing();
            }

            if (game.DisplayBones)
            {
                game.BoneSelector.Render();
            }

            ///hud.Render();

            //-----------

            GraphicsDevice.ResolveBackBuffer(renderTargetRight);

            // change back camera params
            camRotH = camRotH - angle;
            game.Camera.SetRotation(camRotH, camRotV);

            spriteBatch.Begin();
            spriteBatch.Draw(renderTargetLeft, leftRectangle, srcRectangle ,Color.White);
            spriteBatch.Draw(renderTargetRight, rightRectangle, srcRectangle, Color.White);
            spriteBatch.End();
            

            base.Draw(gameTime);
            
        }
        #endregion Draw


        private void InitRenderTargets()
        {
           // System.Diagnostics.Debug.WriteLine("width:"+renderTargetWidth);
           // System.Diagnostics.Debug.WriteLine("Height:"+renderTargetHeight);
            renderTargetLeft = new ResolveTexture2D(GraphicsDevice,
                renderTargetWidth, renderTargetHeight, 1,
                GraphicsDevice.PresentationParameters.BackBufferFormat);

            renderTargetRight = new ResolveTexture2D(GraphicsDevice,
                renderTargetWidth, renderTargetHeight, 1,
                GraphicsDevice.PresentationParameters.BackBufferFormat);
        }
    }
}

Preview :

Spoiler:

Now i'm working on head tracker client connection (maybe with telnet) and doom like keyboard shortcuts to move camera

Here a little tutorial :


[You must be registered and logged in to see this link.]
I hope that this tutorial is not to teach me ? Laughing

The "alpha issue" has nothing to do with "backface culling" !
Solution:
Code:

            // render
            // -----------------
            // This is required .... or save the RenderState
           GraphicsDevice.RenderState.DepthBufferEnable = false;

           spriteBatch.Begin();
           spriteBatch.Draw(renderTargetLeft, leftRectangle, srcRectangle ,Color.White);
           spriteBatch.Draw(renderTargetRight, rightRectangle, srcRectangle, Color.White);
           spriteBatch.End();
           GraphicsDevice.RenderState.DepthBufferEnable = true;

BTW:
Already some ideas about the "select bone in 3D" issue ?
Already some ideas about the "Options-->Camera parameter ..." issue?

XNAaraL
XNALara teacher
XNALara teacher

Male Сообщения : 688
Location : Германия

Back to top Go down

Re: Google cardboard, color-cross

Post by kenran on Fri Feb 20, 2015 6:55 pm

What is this "Select bone in 3D" issue ? ( maybe an other topic )
In Treejs we do it by using ray tracing projection. The first object the ray pass through is the selected bone there's a function for it.

Yeah for the camera dialog it's impossible to type a number. I will fix that.

The video is just the ir header tracking interface construction.
Compatible webcam list :
[You must be registered and logged in to see this link.]

For connecting free tracker the simplest way is to use PPJoy a joystick emulator (not telnet as i said previously ).
Then i created a Game component and i get head angles from gamepad axis :

Code:
yawAngle = (-MathHelper.PiOver4 / 150) * GamePad.GetState(PlayerIndex.One).ThumbSticks.Left.X;
                pitchAngle = (MathHelper.PiOver4 / 150) * GamePad.GetState(PlayerIndex.One).ThumbSticks.Left.Y;
                rollAngle = (MathHelper.PiOver4 / 150) * GamePad.GetState(PlayerIndex.One).ThumbSticks.Right.X;

The hard part is to calculate camera target ( i keep the initial distance but target is no longer the bone ). Something like this :

Code:
//Calculate the relative position of the camera                       
                position = Vector3.Transform(Vector3.Backward, Matrix.CreateFromYawPitchRoll(yawAngle, pitchAngle, rollAngle));
                //Convert the relative position to the absolute position
                position += game.Camera.Target;

                //Calculate a new viewmatrix
                viewMatrix = Matrix.CreateLookAt(position, lookAt, Vector3.Up);

Ok a little video explanation of the effect a try to render :

[You must be registered and logged in to see this link.]

But it sucks with no cardboard scratch


kenran
XNALara guest
XNALara guest

Male Сообщения : 7

Back to top Go down

Re: Google cardboard, color-cross

Post by kenran on Fri Mar 06, 2015 1:09 pm

I'm back !!


There is an update for the Stereoscopic component in order to work with the head tracking system :

Code:

using System;
using System.Collections.Generic;
using System.Linq;
using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Audio;
using Microsoft.Xna.Framework.Content;
using Microsoft.Xna.Framework.GamerServices;
using Microsoft.Xna.Framework.Graphics;
using Microsoft.Xna.Framework.Input;
using Microsoft.Xna.Framework.Media;
using Microsoft.Xna.Framework.Net;
using Microsoft.Xna.Framework.Storage;


namespace XNALara
{
    /// <summary>
    /// This is a game component that implements IUpdateable.
    /// </summary>
    public class StereoVR : Microsoft.Xna.Framework.DrawableGameComponent
    {

        private Game game;
        ResolveTexture2D renderTargetLeft;
        ResolveTexture2D renderTargetRight;
        SpriteBatch spriteBatch;
        int renderTargetWidth, renderTargetHeight;
        Rectangle leftRectangle, rightRectangle, srcRectangle;
        Boolean initialized;

        public StereoVR(Game game)
            : base(game)
        {
            // TODO: Construct any child components here
            this.game = game;
            DrawOrder = 10;
            initialized = false;
            game.Window.ClientSizeChanged += new EventHandler(WindowSizeChanged);

            game.Components.Add(this);
        }

        private void WindowSizeChanged(object sender, EventArgs e)
        {
            initialized = false;
        }

        /// <summary>
        /// Allows the game component to perform any initialization it needs to before starting
        /// to run.  This is where it can query for any required services and load content.
        /// </summary>
        public override void Initialize()
        {
            // TODO: Add your initialization code here
            base.Initialize();
        }

        /// <summary>
        /// Allows the game component to update itself.
        /// </summary>
        /// <param name="gameTime">Provides a snapshot of timing values.</param>
        public override void Update(GameTime gameTime)
        {
            // TODO: Add your update code here
            base.Update(gameTime);
        }

        float camRotH;
        float camRotV;
        float angle;

        #region Draw
        /// <summary>
        /// This is where it all happens. Grabs a scene that has already been rendered,
        /// and uses postprocess magic to add the same view with a different camera angle.
        /// </summary>
        public override void Draw(GameTime gameTime)
        {

            if (!initialized)
            {
                spriteBatch = new SpriteBatch(GraphicsDevice);
                renderTargetWidth = game.Window.ClientBounds.Width;
                renderTargetHeight = game.Window.ClientBounds.Height;
                leftRectangle = new Rectangle(0, 0, renderTargetWidth / 2, renderTargetHeight);
                rightRectangle = new Rectangle(renderTargetWidth / 2, 0, renderTargetWidth / 2, renderTargetHeight);
                srcRectangle = new Rectangle(renderTargetWidth / 4, 0, renderTargetWidth / 2, renderTargetHeight);
                InitRenderTargets();
                initialized = true;
            }

           
            // Resolve the scene into a texture, so we can
            // use it as input data for the stereoscopy.
            GraphicsDevice.ResolveBackBuffer(renderTargetLeft);

            // backup for later
            Matrix m = game.Camera.MatrixView;

            if (game.Camera.MatrixView2 == null)
            {

                // 1 - calculate angle
                // angle is different if object is far or close
                // TAN angle =  (distance neck to eye)/2  /  camera.distance
                angle = (float)(Math.Atan2(0.0325f, game.Camera.Distance)) * 2f;

                // 2 - turn around target           
                camRotH = game.Camera.RotationHorizontal;
                camRotV = game.Camera.RotationVertical;
                camRotH = camRotH + angle;
                game.Camera.SetRotation(camRotH, camRotV);

            }
            else
            {
                // we use freetrack matrix
                game.Camera.MatrixView = game.Camera.MatrixView2;

            }
           
            // render
            // -----------------

            game.GraphicsDevice.Clear(ClearOptions.Target | ClearOptions.DepthBuffer, game.BackgroundColor, 1.0f, 0);
            if (game.BackgroundImage != null)
            {
                game.Renderer.RenderBackgroundImage(0, 0, 0, 0, game.BackgroundImage.ScaleX, game.BackgroundImage.ScaleY);
            }

            game.Renderer.RenderSceneFull(false);

            if (game.IsPostProcessingActive)
            {
                game.Renderer.ApplyPostProcessing();
            }

            if (game.DisplayBones)
            {
                game.BoneSelector.Render();
            }

            [You must be registered and logged in to see this link.]

            //-----------

            GraphicsDevice.ResolveBackBuffer(renderTargetRight);
           
            // change back camera params
            if (game.Camera.MatrixView2 == null)
            {
                camRotH = camRotH - angle;
                game.Camera.SetRotation(camRotH, camRotV);
            }
            else
            {
                game.Camera.MatrixView = m;
            }
                       


            GraphicsDevice.RenderState.DepthBufferEnable = false;

            spriteBatch.Begin();
            spriteBatch.Draw(renderTargetLeft, leftRectangle, srcRectangle ,Color.White);
            spriteBatch.Draw(renderTargetRight, rightRectangle, srcRectangle, Color.White);
            spriteBatch.End();

            GraphicsDevice.RenderState.DepthBufferEnable = true;

            base.Draw(gameTime);
           
        }
        #endregion Draw


        private void InitRenderTargets()
        {
          // System.Diagnostics.Debug.WriteLine("width:"+renderTargetWidth);
          // System.Diagnostics.Debug.WriteLine("Height:"+renderTargetHeight);
            renderTargetLeft = new ResolveTexture2D(GraphicsDevice,
                renderTargetWidth, renderTargetHeight, 1,
                GraphicsDevice.PresentationParameters.BackBufferFormat);

            renderTargetRight = new ResolveTexture2D(GraphicsDevice,
                renderTargetWidth, renderTargetHeight, 1,
                GraphicsDevice.PresentationParameters.BackBufferFormat);
        }
    }
}


The head tracking component (freetrack.cs)
It works even without stereoscopy :

Code:

using System;
using System.Collections.Generic;
using System.Linq;
using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Audio;
using Microsoft.Xna.Framework.Content;
using Microsoft.Xna.Framework.GamerServices;
using Microsoft.Xna.Framework.Graphics;
using Microsoft.Xna.Framework.Input;
using Microsoft.Xna.Framework.Media;
using Microsoft.Xna.Framework.Net;
using Microsoft.Xna.Framework.Storage;
using System.Runtime.InteropServices;



namespace XNALara
{

    class FreeTrackClientDll
    {
        [StructLayout(LayoutKind.Sequential)]
        public struct FreeTrackData
        {
            public int dataid;
            public int camwidth, camheight;
            public Single Yaw, Pitch, Roll, X, Y, Z;
            public Single RawYaw, RawPitch, RawRoll;
            public Single RawX, RawY, RawZ;
            public Single x1, y1, x2, y2, x3, y3, x4, y4;
        }
        [DllImport("FreeTrackClient.dll")]
        public static extern bool FTGetData(ref FreeTrackData data);

        [DllImport("FreeTrackClient.dll")]
        public static extern string FTGetDllVersion();

        [DllImport("FreeTrackClient.dll")]
        public static extern void FTReportID(Int32 name);

        [DllImport("FreeTrackClient.dll")]
        public static extern string FTProvider();

    }




    /// <summary>
    /// This is a game component that implements IUpdateable.
    /// </summary>
    public class Freetrack : Microsoft.Xna.Framework.GameComponent
    {

        private Game game;
        FreeTrackClientDll.FreeTrackData FreeTrackData;

        public Freetrack(Game game)
            : base(game)
        {
            // TODO: Construct any child components here
            this.game = game;
            game.Components.Add(this);
        }

        /// <summary>
        /// Allows the game component to perform any initialization it needs to before starting
        /// to run.  This is where it can query for any required services and load content.
        /// </summary>
        public override void Initialize()
        {
            // TODO: Add your initialization code here
            FreeTrackData = new FreeTrackClientDll.FreeTrackData();
            if (!FreeTrackClientDll.FTGetData(ref FreeTrackData))
            {
                System.Diagnostics.Debug.WriteLine("FTGetData returned false. FreeTrack likely not working.");
                return;
            }

            base.Initialize();
        }

        static float sensibility = 20;
 

        /// <summary>
        /// Allows the game component to update itself.
        /// </summary>
        /// <param name="gameTime">Provides a snapshot of timing values.</param>
        public override void Update(GameTime gameTime)
        {
            UpdateInput();
            base.Update(gameTime);
        }


        /// <summary>
        ///  Main freetrack computing
        /// </summary>
        void UpdateInput()
        {
           
            if (FreeTrackClientDll.FTGetData(ref FreeTrackData))
            {
                 
                   
                Matrix m =
                Matrix.CreateRotationX(-game.Camera.RotationVertical + FreeTrackData.Pitch * sensibility) *
                Matrix.CreateRotationY(game.Camera.RotationHorizontal + FreeTrackData.Yaw * sensibility);
                Vector3 fo = -Vector3.Transform(new Vector3(0, 0, game.Camera.Distance), m);
                Vector3 ta = game.Camera.Position + fo;
                game.Camera.MatrixView = Matrix.CreateLookAt(game.Camera.Position, ta, Vector3.Up);

                // --- support for stereoscopy ---

                // angle is different if object is far or close
                // TAN angle =  (distance neck to eye)/2  /  camera.distance
                float angle = (float)(Math.Atan2(0.0325f, game.Camera.Distance)) * 2f;

                m =
                Matrix.CreateRotationX(-game.Camera.RotationVertical + FreeTrackData.Pitch * sensibility) *
                Matrix.CreateRotationY(game.Camera.RotationHorizontal + angle + FreeTrackData.Yaw * sensibility);

                fo = -Vector3.Transform(new Vector3(0, 0, game.Camera.Distance), m);
                Vector3 po = ta - fo;           

                game.Camera.MatrixView2 = Matrix.CreateLookAt(po, ta, Vector3.Up);
                                   
            } 
           
        }
    }
}

I tried to minimize the original code modifications but this code above need to modify CameraTurnTable.cs :

- Adding a new varaible
- Make matrixView and matrixView2 setable :
Code:

private Matrix matrixView2;

public Matrix MatrixView
        {
            get { return matrixView; }
            set {
                matrixView = value;
            }
        }
        public Matrix MatrixView2
        {
            get { return matrixView2; }
            set {
                matrixView2 = value;
            }
        }


Now how all this works :

1) The Freetrack system :
You need a webcam and an homemade 3 leds tracker . It's all explained here : [You must be registered and logged in to see this link.]

Install freetrack (from the same link above)
Copy freetrackclient.dll to your windows directory ( the dll is in your freetrack install dir , in vista c:\programmes\freetrack)

Launch freetrack, setup your profile and camera ( read the manual pdf ).
Center the skull head, go output tab and uncheck all checkboxes but "freetrack interface".
that's it just run xnalara.


2) How to stream xnalara output to my smartphone :

on your pc :

- download and install kainy and run it.
- go to settings / network you will need the ip and password

On your smartphone :

- download kainy ( ads version works well also )
- connect your phone to your pc via usb ( i didn't tryied wifi )
- see usb tethering here ( [You must be registered and logged in to see this link.] )
- launch kainy
- in server settings enter server ip (192.168.42.113 for me, to find it go to network center and status details of the new phone network) and port (5784). Enter the password ( on your pc is your password)
- click server -> connect
- launch xnalara
Voilà !


kenran
XNALara guest
XNALara guest

Male Сообщения : 7

Back to top Go down

Re: Google cardboard, color-cross

Post by D1m0s on Sun Mar 15, 2015 3:07 pm

TrackIR is great for games like FSX or Warthunder. But why you need to track you head in a posing program? >.>

About Google cardboard... Well.... I've been "playng" with Oculus and Leapmotion for quite a long time. It's fun for user, but for coder it is hell unless you use Unity api. But the worst thing is image quality of Oculus / Cardboard. You can clearly see pixel matrix of the display. Next, you need at least 75 FPS to see smooth image. XNA Framework will allow 60 only. (Oculus DK2 uses display from Note 3)


Anyway I'm interested in how does it look using smartphone. And does it work using USB cable? cyclops So I'm awaiting some review of this "conglomerate" Very Happy

D1m0s
XNALara guest
XNALara guest

Male Сообщения : 2
Location : Far side of the Moon.

Back to top Go down

Re: Google cardboard, color-cross

Post by kenran on Sun Mar 15, 2015 3:27 pm

TrackIR is great for games like FSX or Warthunder. But why you need to track you head in a posing program? >.>
Just for fun Cool . And it's more natural to look with you head than mouse

You don't need 75FPS because it's not a game (no puke). In my duo core 1.6Ghz the rendering reactivity is very good even with the stereo rendering.
The problem is the image compression cpu needs to stream the pc through your phone when you move your head fast.

kenran
XNALara guest
XNALara guest

Male Сообщения : 7

Back to top Go down

Re: Google cardboard, color-cross

Post by D1m0s on Sun Mar 15, 2015 3:39 pm

"Just for fun" - I take this bulletproof argumnet Very Happy I'm going to try it with Oculus. Still, the mouse is the best HID that ever existed. Mouse will be here long after our civilization disappear Very Happy

D1m0s
XNALara guest
XNALara guest

Male Сообщения : 2
Location : Far side of the Moon.

Back to top Go down

Re: Google cardboard, color-cross

Post by Sponsored content


Sponsored content


Back to top Go down

View previous topic View next topic Back to top

- Similar topics

 
Permissions in this forum:
You cannot reply to topics in this forum