[vlc-devel] [PACKAGERS] [RFCv2] PulseAudio removal

[Rémi Denis-Courmont]

   Hello,

On Tue, 05 Apr 2011 10:49:03 +0100, Colin Guthrie <gmane at colin.guthr.ie>
wrote:
>> While the libpulse documentation is much better than the bad joke
>> that the ALSA documentation is, it is still rather terse in many
>> ways, so you have to assume how things just work and that they won't
>> change mysteriously at the next release.
> 
> Can you please give some examples of where things have mysteriously
> changed in a subsequent release?

I wouldn't know about that. But the timing info is explicitly flagged as
subject to change. I spent yet another day of my free time on this. I still
could not figure out how to do proper audio "sync", that is to say with
video.

On a related note, most functions fail to document their return values.
Also some functions and structure parameters lack details in what they
really do. It might be obvious for PulseAudio developers. But I dare say
that they should not be the target *audience*. In particular, what exactly
is the stream "latency"? There are many ways to define latency in the
context of a PA (playback) stream.

> The only things I can think of are the
> buffer metrics where initialising all values to -1 was added as a "try
> to do things with the best power consumption in mind" indicator. This is
> one of the things Lennart mentions as being a little complicated[1] but
> in order to specifically NOT "change things mysteriously at the next
> release"

Yeah... That is a bit silly: There is documentation... in the wiki, but
not in the Doxygen. Luckily I found the wiki article anyway, so that
particular problem did not affect me that much.

>> To make matters worse, it is needlessly complicated. Even Lennart
>> admits it, claiming backward compatibility as an excuse. How stupid
>> an excuse is this?
> 
> OK, so on the one hand you complain about things "changing mysteriously
> at the next release" and now you complain about the goal of maintaining
> backwards compatibility... this has to be a new record for self
> contradiction... it was only separated by one sentence!!

There are different ways to construe backward compatibility. Evidently,
you do need to keep the intricate stuff that you have once exposed. But you
can still simplify things through wrappers around the complicated stuff. If
you mean to remove complexity completely, then indeed you have to break
backward compatibility.

VLC depends on libpulse >= 0.9.22, so we would have been totally fine
depending further "new" libpulse functions.

>> What prevents adding nicer functions around the overly intricate ones?
I
>> could
>> not figure any official *and* stable way to either:
>>  - find the play time (or delay till play time) of the buffer write
>>  offset,
> 
> I would have thought that the Latency section of the documentation was
> quite useful:
> 
> "Since updating the timing info structure usually requires a full
> network round trip and some applications monitor the timing very often
> PulseAudio offers a timing interpolation system. If
> PA_STREAM_INTERPOLATE_TIMING is passed when connecting the stream,
> pa_stream_get_time() and pa_stream_get_latency() will try to interpolate
> the current playback time/latency by estimating the number of samples
> that have been played back by the hardware since the last regular timing
> update. It is especially useful to combine this option with
> PA_STREAM_AUTO_TIMING_UPDATE, which will enable you to monitor the
> current playback time/latency very precisely and very frequently without
> requiring a network round trip every time."
> 
> If that is not sufficient for your needs, then the pa_timing_info struct
> should provide all the info you need.

I tried various things from all interpretations of the "latency" that I
could think of. I could not find any way to correlate the read or the write
offset of the buffer to a VLC timestamp (POSIX monotonic clock), which is
ultimately what I need. Whether it's a bug in libpulse, or my stupidity, I
can't say.

>>  - tell PulseAudio that I want a buffer played at a certain time.
> 
> PulseAudio allows you to write data at any time using pa_stream_write.
> You can use the seek offset and the seek flag to play data at any point
> you like. Just use your sample spec and pa_usec_to_bytes() to work out
> the byte offset and and then write your data.

Absolute seek offset is useless as I don't know the time origin of the
playback on PulseAudio side. Relative seek offset is useless as I do not
know the expected playback latency of the current buffer write offset - or
read offset.

In that respect, the stream "latency" is under specified (on the website
Doxygen) as far as I can tell.

> If you want to go back and rewrite data (e.g. the user has triggered a
> rewind/fastforward, track skip etc.) then you can throw away the data
> you've already written and rewrite it. For low power situations (e.g. on
> tablets, laptops etc.) using large buffers is a good thing (it saves
> power) and thus the "uncommon" action of seeking/skipping can be dealt
> with this way.

I do not (necessarily) want to go back. I want to play my sample at a
certain time. VLC measures times by the monotonic clock and I could not
figure any way to translate that to a PulseAudio buffer offset or target
length or whatever.

> That said, I wasn't aware from the API, that VLC allows

...?

>> With neither of those, how the bloody heck am I supposed to
>> synchronize audio and video?! Oh yeah, there are the "raw"
>> time infos, but they are explicitly subject to changes, and
>> barely intelligible if you are not a PulseAudio developer.
> 
> So why not ask for advice? Why not submit documentation patches after
> getting advice?

Sadly, my mail had evidently an overly harsh tone. There are various
reasons, some personal: my well-known hot blood and big mouth, my ongoing
professional challenges, my annoyance at having wasted 2-3 days of my free
time. There are also other issues, like the fact that you seem not have
read the rest of the thread, despair after 3 years with no signs of
improvement, and the feeling of being taken hostage by PulseAudio and the
Linux distributions (those seem shared by others in the VLC development
team), and some _other_ VLC developers feeling very badly of PulseAudio
anyway.

But asking for help is what I am doing in my own way. And I had already
done it:

* I found Lennart was quite clear when he told me "Yeah I should look at
other [than gstreamer] media players, but then again I don't care" (see
other posts in this thread).
* You have known about our challenges for a while.
* We asked the Ubuntu guys too on a number of occasions.
* We asked PulseAudio bug reporters on our own forums.

Sure, I concede I did not come to the PulseAudio mailing list. For one
thing, I was genuinely afraid that I would be met with arrogance, or simply
be told that this was a VLC problem and out-of-topic. Hey, admittedly, that
is a /procès d'intention/. But still, I did try to contact people.

So as far as I am concerned, nobody helped when I asked nicely. Then I
tried the not so nice ways.

> And your comment about "subject to change" is misleading and FUDy. The
> docs clearly state: "Please note that this structure can be *extended*
> as part of evolutionary API updates at any time in any new release."
> 
> This does not mean it will be changed and break your code, it just means
> that it might be *extended* to add even more information in there.

I do not know what it intends to mean. My reading was, this is subject to
change and if you use it, future versions of PulseAudio might break your
application. If that's not what it meant, I stand corrected. But then I
will claim it needs clarification.

>> Colin, you said it was Lennart right to favor gstreamer. Sure. But if
>> only a
>> few PulseAudio developers understand how this thing work, then his
>> arrogant
>> attitude and dismissal of anything other than gstreamer is just totally

>> intolerable. How else is the "competition" supposed to output audio
>> correctly
>> if only gstreamer gets proper support? In my experience, developers of
>> other
>> OSS system bricks are not so partial, say ALSA, D-Bus, XCB, etc
>> "We have support from most Linux distributions" is so
>> irrelevant.
> 
> Well asking is a good start. I'm very surprised that you, whom I have
> respected as a major player and contributor in the FOSS movement, can be
> so ignorant and deliberately obtuse in dealing with this issue.

Uh???

> Is the PA documentation perfect? No. Could it be improved? Sure. This is
> the case with every open source project I know off. I doubt any project
> will ever be considered "done" in that sense. Everything is about
> evolution and, you more than anyone should appreciate that this process
> of evolution requires engagement and collaboration.
> 
> You have deliberately tried to sit in a darkened corner, and work on
> your own to solve a problem and failed. I don't know why you sat in
> isolation like that. I have seen no posts to the PA mailing list asking
> for advice, I have seen no comments on our IRC channel. It's almost like
> you specifically *want* to fail just so you can badmouth PA and Lennart
> some more.

I don't want PulseAudio to fail, and I do think that ALSA is insufficient.
But mostly, I do not want VLC to fail. And currently, exactly that is
occurring as a consequence of PulseAudio.

>> Mind you, just like Lennart, I don't like to be blamed for other
>> people's
>> issues. So if this stuff is not resolved by VLC 1.2.0 release, I will:
>> 1/ disable the VLC PulseAudio plugin that never worked correctly,
>> 2/ take any measure to ensure VLC can access ALSA or OSS without
>> interference
>> (this probably means killing session PulseAudio if present),
>> 3/ when all else fails, tell the user how we feel about this and whom I
>> think
>> to blame.
> 
> OK, so you basically saying you will cripple the user's system?

No.
(1) is mainly about removing bugs - all the contrary of crippling the 
system.
(2) seems quite impossible as PulseAudio is restarted even on SIGKILL.
(3) is just a waiver of warranty of sorts, and cannot cripple the user's
system any further than it already would be.

> You specifically want to make VLC work separately from the audio device
> preferences that the user has picked in their desktop environment's
> sound preferences GUI.

A number of PulseAudio users have it just because their distributions put
it by default. And a number have it because they think it works fine. Alas,
it does not work fine with VLC currently.

> You specifically want to break the integrated
> volume controls both with multimedia keys on laptops/keyboards and with
> panel applets. You specifically want to make it nigh on impossible for
> people to use Bluetooth headphones. You want to break the ability to
> receive a VoIP call - did I mention that PA will automatically mute (or
> if fully supported, actually pause) a properly tagged Music or Video
> audio stream when a Skype call comes in? Neat huh... oh no, actually you
> simply won't be able to receive any VoIP calls now if you implement your
> change.

No, I don't want that AT ALL. Contrary to some other VLC developers, I am
well aware of the advantages of PulseAudio, be that SW mixing, routing,
policing, etc. That might be because I worked at Nokia on Maemo, which was
heavily reliant on PulseAudio, while other VLC developers have not. But I
find that broken VLC sound is much worse than all of these issues - because
I don't like to be blamed.

(...)
> I can honestly say that I have lost a massive amount of respect for you
> due to the attitude portrayed in this message. I thought you took a very
> principled stand with regards to the GPL and VLC in the AppStore stuff.
> I defended your actions via blog comments because I thought that you
> were standing up for the copyleft principles behind the GPL.

I appreciate the support I got from a very few people including you.
I must admit I fail to see the causality here, and what and why you are
accusing me of?

> Here I see that you really don't care about contributing back to other
> projects or helping things evolve for the greater good, and that is a
> real shame.

Why exactly do you think I spent 3 days of my free time trying to work
things out, and a few hours on this email thread for? No matter how unduly
harshly and wrongly I might have expressed myself, these accusations are
totally unfair.

-- 
Rémi Denis-Courmont
http://www.remlab.net/