[vlc-devel] [PACKAGERS] [RFCv2] PulseAudio removal

[Colin Guthrie]

Hi,

(sorry for removing CC... not sure I'm properly subscribed to the list
so posting via gmane and mixing mails + newsgroups causes it's own
problems!)

'Twas brillig, and Rémi Denis-Courmont at 05/04/11 12:03 did gyre and
gimble:
>    Hello,
> 
> On Tue, 05 Apr 2011 10:49:03 +0100, Colin Guthrie <gmane at colin.guthr.ie>
> wrote:
>>> While the libpulse documentation is much better than the bad joke
>>> that the ALSA documentation is, it is still rather terse in many
>>> ways, so you have to assume how things just work and that they won't
>>> change mysteriously at the next release.
>>
>> Can you please give some examples of where things have mysteriously
>> changed in a subsequent release?
> 
> I wouldn't know about that. But the timing info is explicitly flagged as
> subject to change. I spent yet another day of my free time on this. I still
> could not figure out how to do proper audio "sync", that is to say with
> video.

Well I wouldn't worry about the "subject to change" issue. I'll make a
note to clarify the documentation so it doesn't scare people away from
using this structure.

It is also apparent from your experience that documentation improvements
(perhaps some examples?) would help to ensure people can access this
information in an equivalent way.

> On a related note, most functions fail to document their return values.
> Also some functions and structure parameters lack details in what they
> really do. It might be obvious for PulseAudio developers. But I dare say
> that they should not be the target *audience*. In particular, what exactly
> is the stream "latency"? There are many ways to define latency in the
> context of a PA (playback) stream.

Can you give me some examples here? (not necessarily all of them, but
perhaps just a few and some sort of indication if other functions in a
similar area of the API are also affected). That way I can make a point
of going through them and adding appropriate documentation (or perhaps
getting others to do so - there are several people on the PA mailing
list who submit such documentation updates so in an ideal world I'd ask
them!)

I did happen to notice pa_stream_write() doesn't document it's return
which is pretty bad :s (albeit not that important in terms of the the
likelihood of it causing problems)

>> The only things I can think of are the
>> buffer metrics where initialising all values to -1 was added as a "try
>> to do things with the best power consumption in mind" indicator. This is
>> one of the things Lennart mentions as being a little complicated[1] but
>> in order to specifically NOT "change things mysteriously at the next
>> release"
> 
> Yeah... That is a bit silly: There is documentation... in the wiki, but
> not in the Doxygen. Luckily I found the wiki article anyway, so that
> particular problem did not affect me that much.

I have to agree with you on this point. There are several pages on the
wiki with good information on them (although the LatencyControl page is
the most obviously relevant one for programmers). They should at very
least be linked in with the documentation, if not included directly. I
will endeavour to make that so.



>>> To make matters worse, it is needlessly complicated. Even Lennart
>>> admits it, claiming backward compatibility as an excuse. How stupid
>>> an excuse is this?
>>
>> OK, so on the one hand you complain about things "changing mysteriously
>> at the next release" and now you complain about the goal of maintaining
>> backwards compatibility... this has to be a new record for self
>> contradiction... it was only separated by one sentence!!
> 
> There are different ways to construe backward compatibility. Evidently,
> you do need to keep the intricate stuff that you have once exposed. But you
> can still simplify things through wrappers around the complicated stuff. If
> you mean to remove complexity completely, then indeed you have to break
> backward compatibility.
> 
> VLC depends on libpulse >= 0.9.22, so we would have been totally fine
> depending further "new" libpulse functions.

Can you give me a couple examples here?

We're adding a couple new functions in the 1.0 API to deal with
passthrough streams and I would be happy enough to do as you suggest
with other "legacy" approaches if it's at all possible.

But I don't immediately know which bits you're talking about, so if you
could point me in the right direction I'd really appreciate it.

I guess a utility function to set the buffer metrics to all -1 would be
kinda handy, but I suspect you're meaning a bit more than that?

>> I would have thought that the Latency section of the documentation was
>> quite useful:
>>
>> "Since updating the timing info structure usually requires a full
>> network round trip and some applications monitor the timing very often
>> PulseAudio offers a timing interpolation system. If
>> PA_STREAM_INTERPOLATE_TIMING is passed when connecting the stream,
>> pa_stream_get_time() and pa_stream_get_latency() will try to interpolate
>> the current playback time/latency by estimating the number of samples
>> that have been played back by the hardware since the last regular timing
>> update. It is especially useful to combine this option with
>> PA_STREAM_AUTO_TIMING_UPDATE, which will enable you to monitor the
>> current playback time/latency very precisely and very frequently without
>> requiring a network round trip every time."
>>
>> If that is not sufficient for your needs, then the pa_timing_info struct
>> should provide all the info you need.
> 
> I tried various things from all interpretations of the "latency" that I
> could think of. I could not find any way to correlate the read or the write
> offset of the buffer to a VLC timestamp (POSIX monotonic clock), which is
> ultimately what I need. Whether it's a bug in libpulse, or my stupidity, I
> can't say.

Well, being pragmatic, I'd say that if *you* cannot work it out, then
the documentation needs to be clearer.

As I said in my original reply, I don't expect documentation for any
project to be perfect, but for it to improve, it requires feedback and
engagement.

I'm going to put some time aside at LAC (start of May) and Randa (start
of June) to take a look specifically at this. I will use myself as a
guinea pig as I've not used the API from this side very much.

>>>  - tell PulseAudio that I want a buffer played at a certain time.
>>
>> PulseAudio allows you to write data at any time using pa_stream_write.
>> You can use the seek offset and the seek flag to play data at any point
>> you like. Just use your sample spec and pa_usec_to_bytes() to work out
>> the byte offset and and then write your data.
> 
> Absolute seek offset is useless as I don't know the time origin of the
> playback on PulseAudio side. Relative seek offset is useless as I do not
> know the expected playback latency of the current buffer write offset - or
> read offset.
> 
> In that respect, the stream "latency" is under specified (on the website
> Doxygen) as far as I can tell.

Yes, I can appreciate this needs clarification. I'll see if I can do
that for you with some consultation with others.

>> So why not ask for advice? Why not submit documentation patches after
>> getting advice?
> 
> Sadly, my mail had evidently an overly harsh tone. There are various
> reasons, some personal: my well-known hot blood and big mouth, my ongoing
> professional challenges, my annoyance at having wasted 2-3 days of my free
> time. There are also other issues, like the fact that you seem not have
> read the rest of the thread, despair after 3 years with no signs of
> improvement, and the feeling of being taken hostage by PulseAudio and the
> Linux distributions (those seem shared by others in the VLC development
> team), and some _other_ VLC developers feeling very badly of PulseAudio
> anyway.
> 
> But asking for help is what I am doing in my own way. And I had already
> done it:
> 
> * I found Lennart was quite clear when he told me "Yeah I should look at
> other [than gstreamer] media players, but then again I don't care" (see
> other posts in this thread).
> * You have known about our challenges for a while.
> * We asked the Ubuntu guys too on a number of occasions.
> * We asked PulseAudio bug reporters on our own forums.
> 
> Sure, I concede I did not come to the PulseAudio mailing list. For one
> thing, I was genuinely afraid that I would be met with arrogance, or simply
> be told that this was a VLC problem and out-of-topic. Hey, admittedly, that
> is a /procès d'intention/. But still, I did try to contact people.

I can guarantee that I will do my utmost to help should you ever post to
the mailing list. I try to keep it in good order and probe and ask
questions and ensure anything unhelpful/counter productive is. I very
much doubt any open question asked without prejudice would be treated
with arrogance or derision. The tone of the email is obviously very
important. Open and engaging will elicit a much better response! It's a
very open and friendly list and people have commented to me very
positively on the way it is generally run. Please do not be put off
joining (tho' we will be moving to fd.o soon ish, but I hope any
subscription will be seamlessly migrated - fingers crossed!)

I have to admit to only using VLC relatively infrequently and when I do,
the PA output has never been so terrible it caused me any significant
issues (the buffer metrics are not ideally chosen for network latencies
but that's not a primary focus). While I know it was sub-optimal, I
wasn't aware it was an issue that was so problematic. I apologise for
not getting a fuller picture of things from you in the past. If you
could provide a presee of the problems you encounter that would be great.

As various Ubuntu guys, myself and others will be at LAC, I'll make a
point of trying to get something productive out of that time.


>> And your comment about "subject to change" is misleading and FUDy. The
>> docs clearly state: "Please note that this structure can be *extended*
>> as part of evolutionary API updates at any time in any new release."
>>
>> This does not mean it will be changed and break your code, it just means
>> that it might be *extended* to add even more information in there.
> 
> I do not know what it intends to mean. My reading was, this is subject to
> change and if you use it, future versions of PulseAudio might break your
> application. If that's not what it meant, I stand corrected. But then I
> will claim it needs clarification.

I will ensure it is clarified.


>>> Mind you, just like Lennart, I don't like to be blamed for other
>>> people's
>>> issues. So if this stuff is not resolved by VLC 1.2.0 release, I will:
>>> 1/ disable the VLC PulseAudio plugin that never worked correctly,
>>> 2/ take any measure to ensure VLC can access ALSA or OSS without
>>> interference
>>> (this probably means killing session PulseAudio if present),
>>> 3/ when all else fails, tell the user how we feel about this and whom I
>>> think
>>> to blame.
>>
>> OK, so you basically saying you will cripple the user's system?
> 
> No.
> (1) is mainly about removing bugs - all the contrary of crippling the 
> system.

A laudable goal, but one I think that will not be achieved by this
approach (well, "bugs" could technically be gone, as "intended
behaviour" is not a bug, but the general operation of the users
environment is still busted!)

> (2) seems quite impossible as PulseAudio is restarted even on SIGKILL.

This is, of course, by design. It's autospawning which is enabled by
default. This allows console applications to work happily when PA is not
explicitly started by some kind of init systems (e.g. session launch).
Sensible distros will ensure that they turn autospawn off if the user
disabled PA (certainly I do in Mandriva/Mageia and have advised other
distros in the past to do so also).

There are two very specific ways to interact with PA on a "nice" level.
(1) Via the suspend system. You can specifically request that a given
sink/source is suspended, via the pa_context_suspend_* APIs. This
ensures that PA closes the device and allows another application to use
it. This is what the pasuspender command line tool does.
(2) There is also a DBUS API that allows control of the audio device to
be handed over to another application. This is used e.g. to hand control
over to JACK. This allows JACK to take exclusive control and then for
use to load modules which allow us to output sound via JACK, so the user
should, in theory be (mostly) unaffected.

Using this DBUS protocol is fine, but consideration should also be given
to what state it leaves PA in, e.g. all other applications can no longer
output sound until a suitable sink is loaded into PA. Using the protocol
without considering these issues is obviously not "playing nice" with
the users system.

> (3) is just a waiver of warranty of sorts, and cannot cripple the user's
> system any further than it already would be.

I'm really not sure what you mean here. If you took this action on my
system, it very much would "cripple it more than it already would be".
In what way was my *system* broken before? Some examples would be nice.

>> You specifically want to make VLC work separately from the audio device
>> preferences that the user has picked in their desktop environment's
>> sound preferences GUI.
> 
> A number of PulseAudio users have it just because their distributions put
> it by default. And a number have it because they think it works fine. Alas,
> it does not work fine with VLC currently.

That doesn't mean it doesn't work fine with their *system*. Is VLC
really the judge of what makes a system crippled? As previously stated
any sensible distro provides tools to disable PA if the user so chooses.
If the user chooses to use PA (either because it's the default that the
distro people have chosen and the user has stuck with that, or for any
other reason), then why should VLC take it upon itself to mess about
with that? It's very bad practice IMO to do so. Show warnings or
whatever if you feel it's appropriate, even advise them to disable PA
via their distro's interface to do so, but don't mess with the users' setup.

>> You specifically want to break the integrated
>> volume controls both with multimedia keys on laptops/keyboards and with
>> panel applets. You specifically want to make it nigh on impossible for
>> people to use Bluetooth headphones. You want to break the ability to
>> receive a VoIP call - did I mention that PA will automatically mute (or
>> if fully supported, actually pause) a properly tagged Music or Video
>> audio stream when a Skype call comes in? Neat huh... oh no, actually you
>> simply won't be able to receive any VoIP calls now if you implement your
>> change.
> 
> No, I don't want that AT ALL. Contrary to some other VLC developers, I am
> well aware of the advantages of PulseAudio, be that SW mixing, routing,
> policing, etc. That might be because I worked at Nokia on Maemo, which was
> heavily reliant on PulseAudio, while other VLC developers have not. But I
> find that broken VLC sound is much worse than all of these issues - because
> I don't like to be blamed.

I can appreciate your stance (especially about getting the blame bit),
but I still maintain that the approach you outlined would do all of the
things I said above, but the rhetoric about you "wanting it" is just me
being (overly!) facetious, and I should have toned that back a bit, so
sorry if you took any offence at that bit.

You obviously don't want it. But I genuinely fear that's what will
happen with such an approach.

> (...)
>> I can honestly say that I have lost a massive amount of respect for you
>> due to the attitude portrayed in this message. I thought you took a very
>> principled stand with regards to the GPL and VLC in the AppStore stuff.
>> I defended your actions via blog comments because I thought that you
>> were standing up for the copyleft principles behind the GPL.
> 
> I appreciate the support I got from a very few people including you.
> I must admit I fail to see the causality here, and what and why you are
> accusing me of?

I apologise as the point I was trying to make wasn't clear at all. I
hope it didn't imply something beyond what I meant.

What I was trying to emphasise is that the approach to dealing with the
problem in isolation and then complaining about it to an audience (CCing
me not-with-standing) unrelated to the "source" of the problems, does
not really set the basis for a good community spirit that wants to see
the problems resolved properly. It just allows people to brood and
generally get worked up rather than dealing with the problem.

I drew an analogy with the GPL as one of the goals of the GPL is to aid
sharing, such that if you modify something you share back with the other
people that use it.

So the crux of my point was the the GPL promotes sharing where as your
approach to the problem was the opposite.

I've probably over-explained it now, and it shouldn't really have been a
major point in my original message.

>> Here I see that you really don't care about contributing back to other
>> projects or helping things evolve for the greater good, and that is a
>> real shame.
> 
> Why exactly do you think I spent 3 days of my free time trying to work
> things out, and a few hours on this email thread for? No matter how unduly
> harshly and wrongly I might have expressed myself, these accusations are
> totally unfair.

I apologise. I didn't mean that to sound as over-reaching as it does on
a second reading. It is an unfair accusation as it's written, and I
willingly retract it.

I do, however, maintain that a more open spirit to the problem would
have been more productive. It could have engaged people in the both VLC
and PA communities much more. That said, it's certainly not too late for
that and I'll do my utmost to ensure that it happens still.

All the best.

Col







-- 

Colin Guthrie
gmane(at)colin.guthr.ie
http://colin.guthr.ie/

Day Job:
  Tribalogic Limited [http://www.tribalogic.net/]
Open Source:
  Mageia Contributor [http://www.mageia.org/]
  PulseAudio Hacker [http://www.pulseaudio.org/]
  Trac Hacker [http://trac.edgewall.org/]