[vlc-devel] [PATCH] Flaschen Taschen output: Support for large(r) video sizes
Henner Zeller
h.zeller at acm.org
Sun Apr 26 17:13:06 CEST 2020
Note, I have sent a patch a couple of weeks back here, which allows to
send larger videos, by slicing the packets in horizontal strips. that
are offset. That way, it works around the limited UDP packet size.
https://mailman.videolan.org/pipermail/vlc-devel/2020-March/132387.html
It had some review comments, so I have to adjust the patch, thus it is
not merged yet. I got sucked into some technical Covid19 response
projects so I am currently totally busy with something else. If I
don't get back to vlc FlaschenTaschen it in time, I suggest start with
my patch as a baseline and address the review comments:
https://mailman.videolan.org/pipermail/vlc-devel/2020-March/132582.html
Cheers,
Henner.
On Sun, 26 Apr 2020 at 08:01, Johannes Lechner <git at fudgy.de> wrote:
>
> Hey Rémi,
>
>
> > On 25. Apr 2020, at 11:17, Rémi Denis-Courmont <remi at remlab.net> wrote:
> >
> > Le lauantaina 25. huhtikuuta 2020, 11.06.20 EEST Johannes Lechner a écrit :
> >> This patches contains two modifications to make Flaschen Taschen support
> >> large(r) video sizes: (1) Increase of UDP send buffer, allowing
> >> maximum-sized UDP packets to be sent
> >
> > Increasing the send buffer seems pointless here. Not like it'll magically
> > increase bandwidth. Just wait for buffer space to send the next packet. Either
> > leave the socket in blocking mode, or poll for output.
>
>
> Thanks so much for your guidance, since I am by no means a network guy:
> Just wanted to get my video wall working and contribute the fixes back.
>
> The initial reason why I increased the send buffer size was to get rid of "Message too long” errors during sendmsg.
> Just now I realised that maxing out to the UDP payload will lead to other problems, especially slow downs due to fragmentation on the IP layer.
>
> Please have a look at the updated attached patch:
> - It now splits the frames into multiple UDP packets based on the Ethernet MTU of 1500 bytes (instead of the maximum UDP packet size).
> - Further I’ve implemented poll() in order to be able to drop frames in case the send buffer is full.
>
>
> >> (2) Splitting a frame into multiple
> >> UDP packets in case one frame exceeds the maximum UDP packet size
> >
> > That's good, but I don't really follow how it works here. How does the
> > receiver know which packet belongs to which frame?
>
>
> The Flaschen Taschen protocol is basically “last received packet” wins, i.e. it’s contents remain displayed until a further package is received:
> And the protocol allows you to either update the whole frame displayed or just a sub-section of the frame displayed as specificed through an offset!
>
>
> This offset is sent along in the p6_header (see "snprintf(p6_header…”).
>
>
> e.g. for a 3-by-4 frame where everything fits into one UDP packet:
>
> [UDP Packet #1: Row width=9 (3xRGB), number of rows=4, offset=(0/0)]
> Row #1: XXX-XXX-XXX
> Row #2: XXX-XXX-XXX
> Row #3: XXX-XXX-XXX
> Row #4: XXX-XXX-XXX
>
>
> e.g. for a 3-by-6 frame where two UDP packets would need to be sent:
>
> [UDP Packet #1: Row width=9 (3xRGB), number of rows=4, offset=(0/0)]
> Row #0: XXX-XXX-XXX
> Row #1: XXX-XXX-XXX
> Row #2: XXX-XXX-XXX
> Row #3: XXX-XXX-XXX
>
> [UDP Packet #2: Row width=9 (3xRGB), number of rows=2, offset=(4/0)]
> Row #4: XXX-XXX-XXX
> Row #5: XXX-XXX-XXX
>
>
> Hope this helps?
>
> Johannes
>
>
>
>
>
> >>
> >> The implemention for (2) draws heavily from the "official"
> >> UDPFlaschenTaschen::Send implemention:
> >> https://github.com/hzeller/flaschen-taschen/blob/master/api/lib/udp-flasche
> >> n-taschen.cc ---
> >> modules/video_output/flaschen.c | 113 +++++++++++++++++++-------------
> >> 1 file changed, 66 insertions(+), 47 deletions(-)
> >>
> >> diff --git a/modules/video_output/flaschen.c
> >> b/modules/video_output/flaschen.c index 00e1e66a34..b9f541dc30 100644
> >> --- a/modules/video_output/flaschen.c
> >> +++ b/modules/video_output/flaschen.c
> >> @@ -134,8 +134,9 @@ static int Open(vout_display_t *vd, const
> >> vout_display_cfg_t *cfg,
> >>
> >> /* Ignore any unexpected incoming packet */
> >> setsockopt (fd, SOL_SOCKET, SO_RCVBUF, &(int){ 0 }, sizeof (int));
> >> -
> >> -
> >> + /* Increase send buffer size to allow queuing of several "full" UDP
> >> packets */ + setsockopt (fd, SOL_SOCKET, SO_SNDBUF, &(int){ 16 * 65536
> >> }, sizeof (int)); +
> >> *fmtp = fmt;
> >>
> >> vd->prepare = NULL;
> >> @@ -155,55 +156,73 @@ static void Close(vout_display_t *vd)
> >> free(sys);
> >> }
> >>
> >> +/*
> >> + * Adaptation of UDPFlaschenTaschen::Send implemention
> >> + * (https://github.com/hzeller/flaschen-taschen/)
> >> + */
> >> static void Display(vout_display_t *vd, picture_t *picture)
> >> {
> >> -#ifdef IOV_MAX
> >> - const long iovmax = IOV_MAX;
> >> -#else
> >> - const long iovmax = sysconf(_SC_IOV_MAX);
> >> -#endif
> >> - vout_display_sys_t *sys = vd->sys;
> >> - video_format_t *fmt = &picture->format;
> >> - int result;
> >> -
> >> - char buffer[64];
> >> - int header_len = snprintf(buffer, sizeof(buffer), "P6\n%d %d\n255\n",
> >> - fmt->i_width, fmt->i_height);
> >> - /* TODO: support offset_{x,y,z}? (#FT:...) */
> >> - /* Note the protocol doesn't include any picture order field. */
> >> - /* (maybe add as comment?) */
> >> -
> >> - int iovcnt = 1 + fmt->i_height;
> >> - if (unlikely(iovcnt > iovmax))
> >> - return;
> >> -
> >> - struct iovec iov[iovcnt];
> >> - iov[0].iov_base = buffer;
> >> - iov[0].iov_len = header_len;
> >> -
> >> - uint8_t *src = picture->p->p_pixels;
> >> - for (int i = 1; i < iovcnt; i++)
> >> + video_format_t *format = &picture->format;
> >> + uint8_t *pixels = picture->p->p_pixels;
> >> +
> >> + const int max_data_packet_length = 65507 - 64; // Leave some space for
> >> the P6 header + const size_t row_size = format->i_width * 3;
> >> + const int max_send_height = max_data_packet_length / row_size;
> >> +
> >> + int rows = format->i_height;
> >> + int row_offset = 0;
> >> +
> >> + while(rows)
> >> {
> >> - iov[i].iov_base = src;
> >> - iov[i].iov_len = fmt->i_width * 3;
> >> - src += picture->p->i_pitch;
> >> + // Send as many rows as can fit within one UDP packet
> >> + const int send_height = (rows < max_send_height) ? rows :
> >> max_send_height; +
> >> + char p6_header[64];
> >> + const int p6_header_len = snprintf(p6_header, sizeof(p6_header),
> >> + "P6\n%d %d\n#FT: %d %d
> >> %d\n255\n", + format->i_width,
> >> send_height, + 0, row_offset, 0);
> >> + //msg_Dbg(vd, "p6_header: %s", p6_header`);
> >> +
> >> + const int p6_data_len = send_height * row_size;
> >> +
> >> + // Create and send UDP packet
> >> + {
> >> + struct iovec iov[2];
> >> + iov[0].iov_base = p6_header;
> >> + iov[0].iov_len = p6_header_len;
> >> + iov[1].iov_base = pixels;
> >> + iov[1].iov_len = p6_data_len;
> >> +
> >> + struct msghdr hdr = {
> >> + .msg_name = NULL,
> >> + .msg_namelen = 0,
> >> + .msg_iov = iov,
> >> + .msg_iovlen = 2,
> >> + .msg_control = NULL,
> >> + .msg_controllen = 0,
> >> + .msg_flags = 0 };
> >> +
> >> + int result;
> >> + result = sendmsg(vd->sys->fd, &hdr, 0);
> >> +
> >> + if (result < 0) {
> >> + msg_Err(vd, "sendmsg: error %s in vout display flaschen,
> >> dropping remainder of frame", vlc_strerror_c(errno)); + //
> >> dropping remainder of frame
> >> + break;
> >> + }
> >> + else if (result < (int)(p6_header_len + p6_data_len)) {
> >> + msg_Err(vd, "sendmsg only sent %d bytes in vout display
> >> flaschen, dropping remainder of frame", result); + //
> >> dropping remainder of frame
> >> + break;
> >> + }
> >> + }
> >> +
> >> + pixels += p6_data_len;
> >> +
> >> + rows -= send_height;
> >> + row_offset += send_height;
> >> }
> >> -
> >> - struct msghdr hdr = {
> >> - .msg_name = NULL,
> >> - .msg_namelen = 0,
> >> - .msg_iov = iov,
> >> - .msg_iovlen = iovcnt,
> >> - .msg_control = NULL,
> >> - .msg_controllen = 0,
> >> - .msg_flags = 0 };
> >> -
> >> - result = sendmsg(sys->fd, &hdr, 0);
> >> - if (result < 0)
> >> - msg_Err(vd, "sendmsg: error %s in vout display flaschen",
> >> vlc_strerror_c(errno)); - else if (result < (int)(header_len +
> >> fmt->i_width * fmt->i_height * 3)) - msg_Err(vd, "sendmsg only sent
> >> %d bytes in vout display flaschen", result); - /* we might want to
> >> drop some frames? */
> >> }
> >>
> >> /**
> >
> >
> > --
> > Rémi Denis-Courmont
> > http://www.remlab.net/
>
>
>
> --
> Johannes Lechner · +49 (174) 314 1778 · Skype: thefudgybeaver · Twitter: @fudgybeaver
>
>
>
>
>
>
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel
More information about the vlc-devel
mailing list