[vlc-devel] [RFC PATCH] freetype: using Pango for text layout
Salah-Eddin Shaban
salshaaban at gmail.com
Wed Jan 21 02:31:18 CET 2015
I have investigated the use of Pango in VLC, and it seems simpler than I
thought.
Pango can use FreeType2 as a backend, so we can use it directly in our
freetype module. Here is a first attempt at that.
I have done my best to modify as little as possible and keep the
modifications optional. This is probably sub-optimal. We're using the high
level api of Pango (PangoLayout) for shaping only. Pango can handle the
rendering for us too. And I think using PangoCairo (probably in a separate
module) would prove more efficient and less buggy. Since after setting the
text for the layout we can render everything as a whole to a Cairo surface,
instead of iterating through lines and runs and glyphs.
Anyway, there's still work to be done here regarding error handling and
cleanup, some styles like underline/strikethrough, karaoke, etc. Shadow and
outline. And I still have to see how to modify contribs. I'm not even sure
I have edited configure.ac properly. I'm sorry if I've messed anything up.
I'm really not much of a programmer.
I also haven't figured out yet how to use font attachments with Pango. But
the good thing is that Pango now performs font fallback for us. Try setting
the font to Arial Black, which doesn't have any Arabic glyphs for example,
and Arabic subtitles would still show correctly.
It also fixes another issue I noticed recently, regarding RTL text in
general. You see we're rendering RTL text from left to right, which is fine
since the strings are reversed (by fribidi I suppose). The problem occurs
when the lines wrap, so it's the first part of the text that gets moved to
the next line. And if there are 2 physical lines and they both wrap the
reading order becomes something like:
2
1
4
3
So Pango handles that for us, as well as glyph positioning (base glyphs and
diacritics). Here's a little patch to render diacritics in red and 2
screenshots for comparison:
https://docs.google.com/file/d/0B36ioujDBJZsR28ydHBRWjc5Nm8/edit
https://docs.google.com/file/d/0B36ioujDBJZsNTVLTDVnaFpNSjA/edit
https://docs.google.com/file/d/0B36ioujDBJZsLXFVWG0xa0pKRVk/edit
The red arrows show an example of glyph substitution: 2 diacritic glyphs
are replaced with a single glyph by Pango according to OpenType tables. The
first one is not unreadable, but the second is the official way to render
this combination of diacritics. The blue arrow shows an example of wrong
wrapping of RTL text.
So what do you guys think? Should we go with PangoFT2 or PangoCairo? Same
module or a separate one?
On Wed, Jan 21, 2015 at 3:26 AM, Salah-Eddin Shaban <salshaaban at gmail.com>
wrote:
> ---
> configure.ac | 11 ++
> modules/text_renderer/freetype.c | 297
> +++++++++++++++++++++++++++++++++++++++
> 2 files changed, 308 insertions(+)
>
> diff --git a/configure.ac b/configure.ac
> index 516e9c7..214633f 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -3117,6 +3117,8 @@ AC_ARG_ENABLE(fribidi,
> [ --enable-fribidi fribidi support (default auto)])
> AC_ARG_ENABLE(fontconfig,
> [ --enable-fontconfig fontconfig support (default auto)])
> +AC_ARG_ENABLE(pangoft,
> + [ --enable-pangoft pangoft support (default auto)])
>
> AC_ARG_WITH([default-font],
> AS_HELP_STRING([--with-default-font=PATH],
> @@ -3175,6 +3177,15 @@ if test "${enable_freetype}" != "no"; then
> ],[AC_MSG_WARN([${FRIBIDI_PKG_ERRORS}. Bidirectional support will
> be disabled in FreeType.])])
> fi
>
> + dnl pangoft support
> + if test "${enable_pangoft}" != "no"; then
> + PKG_CHECK_MODULES(PANGOFT, pangoft2, [
> + VLC_ADD_CPPFLAGS([freetype], [${PANGOFT_CFLAGS} -DHAVE_PANGOFT])
> + VLC_ADD_LIBS([freetype], [${PANGOFT_LIBS}])
> + ],[AC_MSG_WARN([${PANGOFT_PKG_ERRORS}. Support for complex
> scripts (Arabic, Farsi, Thai, etc.) will be limited in FreeType.])])
> + fi
> +
> +
> ],[
> have_freetype=no
> AS_IF([test -n "${enable_freetype}"],[
> diff --git a/modules/text_renderer/freetype.c
> b/modules/text_renderer/freetype.c
> index 050ace6..cde3a37 100644
> --- a/modules/text_renderer/freetype.c
> +++ b/modules/text_renderer/freetype.c
> @@ -61,6 +61,13 @@
> # include <fribidi/fribidi.h>
> #endif
>
> +/* Complex Scripts */
> +#if defined(HAVE_PANGOFT)
> +# include <pango/pango.h>
> +# include <pango/pangoft2.h>
> +# include <glib.h>
> +#endif
> +
> /* apple stuff */
> #ifdef __APPLE__
> # include <TargetConditionals.h>
> @@ -138,6 +145,10 @@ static const char *const ppsz_sizes_text[] = {
> #define YUVP_LONGTEXT N_("This renders the font using \"paletized YUV\".
> " \
> "This option is only needed if you want to encode into DVB subtitles" )
>
> +#define PANGO_TEXT N_("Use Pango for text layout")
> +#define PANGO_LONGTEXT N_("Use Pango to layout the text. Required for " \
> + "complex scripts (Arabic, Thai, etc.)")
> +
> static const int pi_color_values[] = {
> 0x00000000, 0x00808080, 0x00C0C0C0, 0x00FFFFFF, 0x00800000,
> 0x00FF0000, 0x00FF00FF, 0x00FFFF00, 0x00808000, 0x00008000, 0x00008080,
> @@ -231,6 +242,10 @@ vlc_module_begin ()
>
> add_bool( "freetype-yuvp", false, YUVP_TEXT,
> YUVP_LONGTEXT, true )
> +#ifdef HAVE_PANGOFT
> + add_bool( "freetype-pango", false, PANGO_TEXT,
> + PANGO_LONGTEXT, true )
> +#endif
> set_capability( "text renderer", 100 )
> add_shortcut( "text" )
> set_callbacks( Create, Destroy )
> @@ -271,6 +286,9 @@ struct line_desc_t
>
> *****************************************************************************/
> struct filter_sys_t
> {
> +#ifdef HAVE_PANGOFT
> + PangoFontMap *p_fontmap;
> +#endif
> FT_Library p_library; /* handle to library */
> FT_Face p_face; /* handle to face object */
> FT_Stroker p_stroker; /* handle to path stroker object */
> @@ -1148,6 +1166,268 @@ static void BBoxEnlarge( FT_BBox *p_max, const
> FT_BBox *p )
> p_max->yMax = __MAX(p_max->yMax, p->yMax);
> }
>
> +#ifdef HAVE_PANGOFT
> +static int ProcessLinesPangoFT2( filter_t *p_filter,
> + line_desc_t **pp_lines,
> + FT_BBox *p_bbox,
> + int *pi_max_face_height,
> + uni_char_t *psz_text,
> + text_style_t **pp_styles,
> + uint32_t *pi_k_dates,
> + int i_len )
> +{
> + filter_sys_t *p_sys = p_filter->p_sys;
> + *pi_max_face_height = 0;
> + *pp_lines = NULL;
> + line_desc_t **pp_line_next = pp_lines;
> +
> + FT_BBox bbox = { .xMin = INT_MAX, .yMin = INT_MAX,
> + .xMax = INT_MIN, .yMax = INT_MIN };
> +
> + int i_face_height_previous = 0, i_base_line = 0;
> + //const text_style_t *p_previous_style = NULL;
> + //FT_Face p_face = NULL;
> +
> + /*
> + * Convert to UTF-8 for Pango. psz_text is not recognized as
> + * UCS-4 unless the byte order is reversed.
> + */
> + for( int i = 0; i < i_len; ++i )
> + {
> + guchar *p = ( guchar * ) psz_text + i * sizeof( *psz_text );
> + guchar ch0 = *( p ); guchar ch1 = *( p + 1 );
> + guchar ch2 = *( p + 2 ); guchar ch3 = *( p + 3 );
> +
> + *( p + 0 ) = ch3; *( p + 1 ) = ch2;
> + *( p + 2 ) = ch1; *( p + 3 ) = ch0;
> + }
> +
> + gsize l_bytes_read = 0, l_bytes_written = 0;
> + gchar *psz_utf8 = g_convert( ( gchar * ) psz_text, i_len * sizeof(
> *psz_text ),
> + "UTF-8", "UCS-4", &l_bytes_read,
> &l_bytes_written, 0 );
> + if( !psz_utf8 ) {
> + msg_Err( p_filter, "Failed to convert text to UTF-8" );
> + return VLC_EGENERIC;
> + }
> +
> + PangoFontMap *p_fm;
> + PangoContext *p_context;
> + PangoFontDescription *p_font_desc;
> + PangoLayout *p_layout;
> +
> + p_fm = p_sys->p_fontmap;
> + p_context = pango_font_map_create_context(p_fm);
> + if( !p_context )
> + {
> + msg_Err( p_filter, "Failed to create a Pango context" );
> + g_free( psz_utf8 );
> + return VLC_EGENERIC;
> + }
> + pango_context_set_base_dir( p_context, PANGO_DIRECTION_LTR );
> + p_font_desc = pango_font_description_new();
> + pango_font_description_set_family( p_font_desc,
> p_sys->style.psz_fontname );
> + pango_font_description_set_absolute_size( p_font_desc,
> p_sys->style.i_font_size * PANGO_SCALE );
> + pango_context_set_font_description( p_context, p_font_desc );
> + p_layout = pango_layout_new( p_context );
> + pango_layout_set_width( p_layout, ( int )
> p_filter->fmt_out.video.i_visible_width
> + * PANGO_SCALE );
> + pango_layout_set_height( p_layout, ( int )
> p_filter->fmt_out.video.i_visible_height
> + * PANGO_SCALE );
> + pango_layout_set_auto_dir( p_layout, false );
> +
> + /* Set attributes that affect text shaping */
> + PangoAttrList *p_list = pango_attr_list_new();
> + text_style_t *p_style = pp_styles[0];
> + gchar *p0 = psz_utf8, *p1 = psz_utf8;
> + for( int i = 0; i < i_len; ++i ) {
> + if( !FaceStyleEquals( p_style, pp_styles[i] ) ||
> + p_style->i_font_size != pp_styles[i]->i_font_size ||
> + i == i_len - 1 )
> + {
> + p_font_desc = pango_font_description_new();
> + pango_font_description_set_family( p_font_desc,
> p_style->psz_fontname );
> + pango_font_description_set_absolute_size( p_font_desc,
> + p_style->i_font_size
> * PANGO_SCALE );
> +
> + PangoAttribute *p_attr = pango_attr_font_desc_new(
> p_font_desc );
> + p_attr->start_index = p0 - psz_utf8;
> + p_attr->end_index = p1 - psz_utf8;
> + pango_attr_list_insert( p_list, p_attr );
> + p_style = pp_styles[i];
> + p0 = p1;
> + }
> + p1 = g_utf8_next_char(p1);
> + }
> +
> + /* Perform text shaping */
> + pango_layout_set_attributes( p_layout, p_list );
> + pango_layout_set_text( p_layout, psz_utf8, l_bytes_written);
> +
> + /* Now the text has been laid out, fill our pp_lines */
> + GSList *p_layout_lines = pango_layout_get_lines( p_layout );
> + while( p_layout_lines ) {
> + PangoLayoutLine *p_layout_line = ( PangoLayoutLine * )
> p_layout_lines->data;
> + int i_glyph_count = 0;
> + GSList *p_runs = p_layout_line->runs;
> + while( p_runs ) {
> + PangoLayoutRun *p_run = ( PangoLayoutRun * ) p_runs->data;
> + PangoGlyphString *p_glyphs = p_run->glyphs;
> + i_glyph_count += p_glyphs->num_glyphs;
> + p_runs = g_slist_next( p_runs );
> + }
> +
> + line_desc_t *p_line = i_glyph_count > 0 ? NewLine( i_glyph_count
> ) : NULL;
> + if( p_line ) p_line->i_character_count = i_glyph_count;
> + FT_Vector pen = { .x = 0, .y = 0 };
> + int i_font_width = p_sys->style.i_font_size;
> + int i_face_height = 0;
> + FT_BBox line_bbox = { .xMin = INT_MAX, .yMin = INT_MAX,
> + .xMax = INT_MIN, .yMax = INT_MIN };
> +
> + /*
> + * A cluster denotes a base glyph + its diacritics (accent
> glyphs). To apply
> + * remaining styles to glyphs we need to iterate through the
> clusters. That
> + * seems to be the only way for us to know which source text
> characters correspond
> + * to which glyphs. There's no 1:1 mapping here since Pango may
> have performed
> + * glyph substitutions according to OpenType tables.
> + */
> + p_runs = p_layout_line->runs;
> + int i_line_index = 0;
> + while( p_runs ) {
> + PangoLayoutRun *p_run = ( PangoLayoutRun * ) p_runs->data;
> + PangoGlyphString *p_glyphs = p_run->glyphs;
> + PangoGlyphItemIter cluster_iter;
> + gboolean b_rtl = p_run->item->analysis.level % 2;
> + gboolean b_have_cluster = b_rtl ?
> + pango_glyph_item_iter_init_end( &cluster_iter, p_run,
> psz_utf8 ) :
> + pango_glyph_item_iter_init_start( &cluster_iter, p_run,
> psz_utf8 );
> +
> + for ( ;
> + b_have_cluster;
> + b_have_cluster = b_rtl ?
> + pango_glyph_item_iter_prev_cluster ( &cluster_iter ) :
> + pango_glyph_item_iter_next_cluster ( &cluster_iter ) )
> + {
> + int i_glyph_index = b_rtl ? cluster_iter.end_glyph + 1 :
> + cluster_iter.start_glyph;
> + while( b_rtl ? ( i_glyph_index !=
> cluster_iter.start_glyph + 1) :
> + ( i_glyph_index != cluster_iter.end_glyph
> ) )
> + {
> + FT_Glyph glyph;
> + FT_BBox glyph_bbox;
> + FT_Glyph outline;
> + FT_BBox outline_bbox;
> + FT_Glyph shadow;
> + FT_BBox shadow_bbox;
> +
> + FT_Face p_face = pango_fc_font_lock_face(
> + PANGO_FC_FONT( p_run->item->analysis.font ) );
> + i_face_height = __MAX( i_face_height,
> + FT_CEIL(FT_MulFix(p_face->height,
> p_face->size->metrics.y_scale)));
> +
> + FT_Vector pen_new;
> + /* Divide by 16 to convert from PangoUnits to
> FreeType's 26.6 format */
> + pen_new.x = pen.x +
> p_glyphs->glyphs[i_glyph_index].geometry.x_offset / 16;
> + pen_new.y = pen.y -
> p_glyphs->glyphs[i_glyph_index].geometry.y_offset / 16;
> +
> + FT_Vector pen_shadow = {
> + .x = pen_new.x + p_sys->f_shadow_vector_x *
> (i_font_width << 6),
> + .y = pen_new.y + p_sys->f_shadow_vector_y *
> (i_font_width << 6),
> + };
> +
> + if( GetGlyph( p_filter,
> + &glyph, &glyph_bbox,
> + &outline, &outline_bbox,
> + &shadow, &shadow_bbox,
> + p_face,
> p_glyphs->glyphs[i_glyph_index].glyph, 0,
> + &pen_new, &pen_shadow) )
> + {
> + p_line->p_character[i_line_index++] =
> (line_character_t) {0};
> + pango_fc_font_unlock_face(
> + PANGO_FC_FONT( p_run->item->analysis.font ) );
> + ++i_glyph_index;
> + continue;
> + }
> +
> + FixGlyph( glyph, &glyph_bbox, p_face, &pen_new );
> + if( outline )
> + FixGlyph( outline, &outline_bbox, p_face,
> &pen_new );
> + if( shadow )
> + FixGlyph( shadow, &shadow_bbox, p_face,
> &pen_shadow );
> +
> + pango_fc_font_unlock_face(
> + PANGO_FC_FONT( p_run->item->analysis.font ) );
> +
> + FT_BBox line_bbox_new = line_bbox;
> + BBoxEnlarge( &line_bbox_new, &glyph_bbox );
> + if( outline )
> + BBoxEnlarge( &line_bbox_new, &outline_bbox );
> + if( shadow )
> + BBoxEnlarge( &line_bbox_new, &shadow_bbox );
> +
> + gchar *p = &psz_utf8[ cluster_iter.start_index ];
> + int i_index_in_chars = g_utf8_strlen( psz_utf8, p -
> psz_utf8 );
> + p_style = pp_styles[ i_index_in_chars ];
> + p_line->p_character[ i_line_index++ ] =
> (line_character_t) {
> + .p_glyph = (FT_BitmapGlyph) glyph,
> + .p_outline = (FT_BitmapGlyph) outline,
> + .p_shadow = (FT_BitmapGlyph) shadow,
> + .i_color = p_style->i_font_color |
> p_style->i_font_alpha << 24,
> + .i_line_offset = 0,
> + .i_line_thickness = 0,
> + };
> +
> + /*
> + * We're now using Pango for glyph positioning. If 2
> glyphs should
> + * be rendered on top of one another, the width of
> the first will
> + * be 0. So we no longer have to check for diacritics
> or zero-width
> + * spaces, etc.
> + */
> + pen.x = pen.x +
> p_glyphs->glyphs[i_glyph_index].geometry.width / 16;
> + line_bbox = line_bbox_new;
> + ++i_glyph_index;
> + }
> + }
> + p_runs = g_slist_next(p_runs);
> + }
> +
> + if( i_face_height_previous > 0 )
> + i_base_line += __MAX(i_face_height, i_face_height_previous);
> + if( i_face_height > 0 )
> + i_face_height_previous = i_face_height;
> +
> +
> + /* Update the line bbox with the actual base line */
> + if (line_bbox.yMax > line_bbox.yMin) {
> + line_bbox.yMin -= i_base_line;
> + line_bbox.yMax -= i_base_line;
> + }
> + BBoxEnlarge( &bbox, &line_bbox );
> +
> + if( p_line )
> + {
> + p_line->i_width = __MAX(line_bbox.xMax - line_bbox.xMin, 0);
> + p_line->i_base_line = i_base_line;
> + p_line->i_height = __MAX(i_face_height,
> i_face_height_previous);
> + *pp_line_next = p_line;
> + pp_line_next = &p_line->p_next;
> +
> + }
> + *pi_max_face_height = __MAX( *pi_max_face_height, i_face_height );
> +
> + p_layout_lines = g_slist_next(p_layout_lines);
> + }
> +
> + g_object_unref( p_layout );
> + pango_font_description_free( p_font_desc );
> + g_object_unref( p_context );
> + if( psz_utf8 ) g_free( psz_utf8 );
> +
> + *p_bbox = bbox;
> + return VLC_SUCCESS;
> +}
> +#endif
> +
> static int ProcessLines( filter_t *p_filter,
> line_desc_t **pp_lines,
> FT_BBox *p_bbox,
> @@ -1767,9 +2047,20 @@ static int RenderCommon( filter_t *p_filter,
> subpicture_region_t *p_region_out,
>
> if( !rv && i_text_length > 0 )
> {
> +#ifdef HAVE_PANGOFT
> + if( var_InheritBool( p_filter, "freetype-pango" ) )
> + rv = ProcessLinesPangoFT2( p_filter,
> + &p_lines, &bbox,
> &i_max_face_height,
> + psz_text, pp_styles,
> pi_k_durations, i_text_length );
> + else
> + rv = ProcessLines( p_filter,
> + &p_lines, &bbox, &i_max_face_height,
> + psz_text, pp_styles, pi_k_durations,
> i_text_length );
> +#else
> rv = ProcessLines( p_filter,
> &p_lines, &bbox, &i_max_face_height,
> psz_text, pp_styles, pi_k_durations,
> i_text_length );
> +#endif
> }
>
> p_region_out->i_x = p_region_in->i_x;
> @@ -1865,6 +2156,9 @@ static int Init_FT( vlc_object_t *p_this,
> filter_sys_t *p_sys = p_filter->p_sys;
>
> /* */
> +#ifdef HAVE_PANGOFT
> + p_sys->p_fontmap = pango_ft2_font_map_new();
> +#endif
> int i_error = FT_Init_FreeType( &p_sys->p_library );
> if( i_error )
> {
> @@ -2058,6 +2352,9 @@ static void Destroy_FT( vlc_object_t *p_this )
> FT_Stroker_Done( p_sys->p_stroker );
> FT_Done_Face( p_sys->p_face );
> FT_Done_FreeType( p_sys->p_library );
> +#ifdef HAVE_PANGOFT
> + g_object_unref( p_sys->p_fontmap );
> +#endif
> }
>
>
> /*****************************************************************************
> --
> 1.9.1
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20150121/f24afbe1/attachment.html>
More information about the vlc-devel
mailing list