[vlc-devel] [PATCH 11/18] input: use vlc_viewpoint_from/to_euler
Alexandre Janniaux
ajanni at videolabs.io
Thu Apr 1 14:15:01 UTC 2021
Hi,
Let´s start again on a more consistent explanation:
1/ what is gimbal lock
2/ why this is needed
I´ll start with the explanation you read from SO:
> "Euler Angles" you can think of as a function (S1)3→SO3
> or R3→SO3. The derivative of this function does not always
> have rank 3, so you have degenerate submanifolds where the
> function is many-to-one. In this special case that's
> called "gimbal lock".
A more direct explanation is to reference Polar Coordinates
because it´s a well known engineering concept. Without being
formal:
All point in space with a given origin can be described by
it´s radius and the yaw/pitch angles, and this representation
is unique except for the origin point.
In this representation, for the origin. the radius is 0, but
there´s an infinity of matching yaw/pitch (all actually) that
alias this point.
This example is interesting for two reasons:
- if you don´t need the radius, ie if you only want direction
and orientation (or just orientation vector/eye vector),
then you only need yaw and pitch.
- it shows what is the concept of aliasing.
When you get back to the Euler angles, it´s basically made
of three distinct rotation. If you take our usual rotation
order convention, and look up or down (+- 90°), then the
rotation axis for yaw and roll are matching, so you also
have an infinity of possible rotation (you can increase yaw
as much as you want as long as you compensate with the roll
value basically).
When you convert this into quaternion, you basically convert
this into a single orientation vector and a single roll value
which, as opposed to Euler angles, are able to uniquely
identify every orientation possible.
When you convert back from quaternion into Euler angles, you
thus have an infinite number of representation possible for
the Euler angles, making it unusable for computation. What it
means exactly is that if you get too close to that point, the
yaw/roll will become totally different from what you had
previously, so when you get out of this point, you´ll have
your head turn (like if you changed roll) and not be aligned
with your front (like if you changed yaw), much like the
polar situation where (0, 10, 20) is the same point as
(0, 1, 20), ie. where shrinking to 0 make you lose your
orientation.
So now, you could mathematically come as close as possible
to these singularity points, but with floating points it
basically means that the closer you are to this point, the
bigger the errors you make. You can draw a mathematical
limit by saying «I don´t want to go over some specified
error» but tbh, computing this is far from being in my
skillset (and it´s painful, takes a lot of time, just for
some specified value) so I cannot do that. You won´t have
a closed limit that you can use here, just like you cannot
approximate 1/x with x as low as you want on computers with
just a single fixed-size representation.
So yes, this approximate the relative updates to preserve
the Euler angles values when they are represented as
quaternion. This is done like this because the hotkeys and
inputs are doing relative updates at first, but there´s no
approximation if you´re doing absolute updates. Basically
if you want to do relative updates of the yaw/pitch/roll
values, you need to keep the yaw/pitch/roll values and
regenerate the whole result. That´s what I mean behind:
> Since hotkeys and input were not handling the problem themselves
> and were supplying relative transformation, such relative rotations
> are clipped to avoid the poles.
And thus the final solution is to keep them as absolute.
Said otherwise, the yaw/pitch/roll paradigm only makes sense
in places where you actually have a yaw/pitch/roll. When
you´re not (eg. an HMD headset, in which it doesn´t exist
since you cannot match the reality with the point space
since you cannot name the north and south points), then you
shouldn´t use that paradigm. Thus, the patchset basically use
a more generic orientation mechanism for the core and does
the bare minimum to keep the current experience.
Then, you mentioned that I didn´t answer your question about
HMD headsets. What you describe is _exactly_ why I made this
patchset, and it´s pretty straightforward with all this
background and just the «data»:
- conversion between Euler angles lead to singularities
- HMD headset are generating quaternions
- if you need Euler angle in the core, you need to convert
the quaternions into Euler angles
- thus, you have singularities and cannot look at the top
or bottom without exceptional seasickness afterwards.
In a more general fashion, like mentioned above, you also
need to match the reality with the orientation system you
use and reality doesn´t use Euler angles, only engineers
are.
And finally, there´s basically 12 different possibles
rotation system that you can create with Euler angles (with
different poles). So there are points that are unique at
some coordinates in one systems but aren´t in another system,
so rotations are a bit more tricky than just angles.
Euler angle systems are also pretty inconvenient when you
want to convert between systems (like, from the mp4-specified
coordinate system to another that would be more convenient
for the end user). To do that, you basically need an
intermediate representation that is sufficiently non-biased
to represent the orientation, which basically means quaternion
or any other isomorphism. So having quaternion in the core
for representation of the orientation is much more convenient
than Euler angles.
I hope it answers both your questions on the nature of the
value and the reasons behind using quaternions here. In the
meantime, I reviewed some comments with Romain, which were
wrong (V_13/31 shift) and the case for handling the
singularity didn´t match the computations in the comments and
were incorrect, so I´m fixing those.
Regards,
--
Alexandre Janniaux
Videolabs
On Thu, Apr 01, 2021 at 02:19:03PM +0200, Steve Lhomme wrote:
> On 2021-04-01 13:24, Alexandre Janniaux wrote:
> > Hi,
> >
> > On Thu, Apr 01, 2021 at 12:45:35PM +0200, Steve Lhomme wrote:
> > > On 2021-04-01 10:51, Alexandre Janniaux wrote:
> > > > Hi,
> > > > On Thu, Apr 01, 2021 at 08:02:10AM +0200, Steve Lhomme wrote:
> > > > > On 2021-03-31 11:25, Alexandre Janniaux wrote:
> > > > > > From: Alexandre Janniaux <alexandre.janniaux at gmail.com>
> > > > > >
> > > > > > ... and also clip pitch to avoid singularities in locations where a
> > > > >
> > > > > It looks like a commit title that was cut by GitHub.
> > > >
> > > > Not really, github doesn´t wrap at 38 characters. I just
> > > > didn´t duplicate the short commit message because it brings
> > > > no value here.
> > > >
> > > > > > relative viewpoint update is used. Otherwise, conversions between
> > > > > > viewpoints and euler angles might lead to corrupted values and blinking
> > > > > > pitch angles.
> > > > > > ---
> > > > > > src/input/input.c | 22 ++++++++++++++++++----
> > > > > > 1 file changed, 18 insertions(+), 4 deletions(-)
> > > > > >
> > > > > > diff --git a/src/input/input.c b/src/input/input.c
> > > > > > index 69552bc3e5..b134d4f097 100644
> > > > > > --- a/src/input/input.c
> > > > > > +++ b/src/input/input.c
> > > > > > @@ -1758,6 +1758,10 @@ static void ControlNav( input_thread_t *p_input, int i_type )
> > > > > > if( b_viewpoint_ch && viewpoint_updated )
> > > > > > {
> > > > > > + /* We need to clip the pitch here to avoid singularity poles, otherwise
> > > > > > + * the next vlc_viewpoint_to_euler might reverse the pitch. */
> > > > > > + pitch = VLC_CLIP(pitch, -85.f, 85.f);
> > > > >
> > > > > These arbitrary values should have a name. And why not use -90, 90 ? In
> > > > > Matroska you can have pitch values of -90 to 90, both included. Same for MP4
> > > > > [1]. Does a pitch of 270 turns to 85 ?
> > > > >
> > > > > [1] https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md
> > > >
> > > > They are arbitrary values, so well, it´s quite hard to name them.
> > >
> > > If they are used there must be a reason, that reason should be in the name.
> > >
> > > > They are basically the closest values to the singularity points
> > > > where conversion to/from euler angles doesn´t explode. This is
> > > > explained in the description of the commit.
> > >
> > > I see an explanation on possible consequences, why 85 is the proper value.
> > > In fact I doubt it's a proper value. 90 sounds more like it would be a
> > > mathematical limit. But then this value should be checked when the math will
> > > fail, not on any incoming values.
> >
> > Because that´s explained in the comment above, and that´s a
> > hand-tested value obtained by checking when the math fails.
>
> That sounds very suspicious. Either it's a mathematical limitation and then
> there should be a formula to get the actual value or it's an implementation
> detail and should be explained as such.
>
> > I can use an additional MAX_PITCH/MIN_PITCH constant but even
> > then this value depends on the rotation convention used in the
> > computation so it doesn´t even directly make sense except for
> > the sake of giving a name.
>
> If giving a name to a value makes less sense, there is a problem.
>
> > > More generally you are preventing the input from going to 90 degrees pitch
> > > which might be the user's intention. If the quaternions can't handle such a
> > > position then I don't see any use for them. If they do, this clipping should
> > > not happen at all.
> >
> > You don´t understand the issue here, quaternions can represent this
> > orientation. But Euler angles (and Euler quaternions by isomorphism)
> > cannot and never could [1].
>
> Indeed, I don't understand. You're saying a value of 90 cannot be converted
> to quarternion even though it does exist in Euler (see spherical videos
> specs above), and even if the value can be represented in quaternion.
>
> > The issue happens in situation where you actually do conversion from
> > and to euler angles.
> >
> > [1]: https://math.stackexchange.com/questions/8980/euler-angles-and-gimbal-lock#8984
>
> You lost me at "degenerate submanifolds". I do understand any of this text.
>
> > > If you have a headset/HMD and you move your head up, do you think it's a
> > > good idea to stop following the movement if it goes above 85 degrees
> > > vertically ? And then restart moving the image when it's over 95 degrees ?
> >
> > I refer you to the cover letter:
> >
> > Euler angles have singularity issues, which in the convention
> > we chosed happens at the north and south poles. It didn't matter
> > when using usual yaw/pitch controllers since there are not a lot
> > of way to do this, but it definitively matters when the orientation
> > is already described by a quaternion. In particular, phone devices
> > or HMD headset with gyroscopes can easily reach the north and south
> > poles while having a different roll-like rotation, so can trigger
> > those bugs.
> >
> > Using quaternions from the start won't trigger those issues in
> > cases where the control is not done through the euler-angle-like
> > paradigm, while still allowing clients to use it as long as they
> > handle the problem themselves.
> >
> > Since hotkeys and input were not handling the problem themselves
> > and were supplying relative transformation, such relative rotations
> > are clipped to avoid the poles.
>
> I read that. That still doesn't answer my question. From what I understand
> you want to use quaternions when doing the math to add new angles to the
> viewpoint to avoid the gimbal lock. But that doesn't say why doing so needs
> to clip the input values we can change at a time. It seems like we're
> replacing one limitation by another.
>
> > > > We can support pitch with singularity values only if there is no
> > > > singularities, meaning only if we don´t have roll, but it would
> > > > mean that we´d have multiple conversion function for the two
> > > > convention used and flag to indicate which convention is in use.
> > > >
> > > > Note also that this is the input code for handling controls and
> > > > for relative updates of the viewpoint, so it has nothing to do
> > > > immediately with the original pose. It´s also converting from
> > > > Euler quaternions so initial pitch values don´t really make
> > > > sense at this point, a pitch of 270 doesn´t exist in this
> > > > convention since it would alias other pitch/yaw values.
> > > >
> > > > This patch does change the previous behaviour, since previous
> > > > behaviour was not clipped. But because there was no conversion
> > > > before it could just work as long as the user didn´t change roll.
> > > >
> > > > Regards,
> > > > --
> > > > Alexandre Janniaux
> > > > Videolabs
> > > >
> > > > > > +
> > > > > > priv->viewpoint_changed = true;
> > > > > > vlc_viewpoint_from_euler( &priv->viewpoint, yaw, pitch, roll );
> > > > > > ViewpointApply( p_input );
> > > > > > @@ -2148,10 +2152,20 @@ static bool Control( input_thread_t *p_input,
> > > > > > else
> > > > > > {
> > > > > > priv->viewpoint_changed = true;
> > > > > > - priv->viewpoint.yaw += param.viewpoint.yaw;
> > > > > > - priv->viewpoint.pitch += param.viewpoint.pitch;
> > > > > > - priv->viewpoint.roll += param.viewpoint.roll;
> > > > > > - priv->viewpoint.fov += param.viewpoint.fov;
> > > > > > + float previous[3], update[3];
> > > > > > + vlc_viewpoint_to_euler(&priv->viewpoint, &previous[0],
> > > > > > + &previous[1], &previous[2]);
> > > > > > + vlc_viewpoint_to_euler(¶m.viewpoint, &update[0],
> > > > > > + &update[1], &update[2]);
> > > > > > +
> > > > > > + /* We need to clip the pitch here to avoid singularity poles,
> > > > > > + * otherwise the next vlc_viewpoint_to_euler might reverse
> > > > > > + * the pitch. */
> > > > > > + vlc_viewpoint_from_euler(&priv->viewpoint,
> > > > > > + previous[0] + update[0],
> > > > > > + VLC_CLIP(previous[1] + update[1], -85.f, 85.f),
> > > > > > + previous[2] + update[2]);
> > > > > > + priv->viewpoint.fov += param.viewpoint.fov;
> > > > > > }
> > > > > > ViewpointApply( p_input );
> > > > > > --
> > > > > > 2.31.0
> > > > > >
> > > > > > _______________________________________________
> > > > > > vlc-devel mailing list
> > > > > > To unsubscribe or modify your subscription options:
> > > > > > https://mailman.videolan.org/listinfo/vlc-devel
> > > > > >
> > > > > _______________________________________________
> > > > > vlc-devel mailing list
> > > > > To unsubscribe or modify your subscription options:
> > > > > https://mailman.videolan.org/listinfo/vlc-devel
> > > > _______________________________________________
> > > > vlc-devel mailing list
> > > > To unsubscribe or modify your subscription options:
> > > > https://mailman.videolan.org/listinfo/vlc-devel
> > > >
> > > _______________________________________________
> > > vlc-devel mailing list
> > > To unsubscribe or modify your subscription options:
> > > https://mailman.videolan.org/listinfo/vlc-devel
> > _______________________________________________
> > vlc-devel mailing list
> > To unsubscribe or modify your subscription options:
> > https://mailman.videolan.org/listinfo/vlc-devel
> >
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel
More information about the vlc-devel
mailing list