[vlc-devel] crash of x264 encoder in windows

[Hannes Domani]

Hello

This are the various problems I had:
 1) crash on x264_encoder_open()
    reason: pthread_win32_process_attach_np() isn't called because
      PTW32_STATIC_LIB isn't set when compiling libx264_plugin
      I didn't find out who should even set it, probably some small
      configure issue.
    workaround: #define PTW32_STATIC_LIB somewhere before #include <pthread.h>
 2) there are some warnings about missing defines
    solution: #include <sched.h> before #include <pthread.h>
 3) crash on x264_encoder_encode()
    reason: this one was a bit trickier to find; i saw that the order
      of functions called was like this:
       - Open()
       - Close()
       - Open()
       - Encode() -> crash
      I was wondering why x264 was initiated twice, but since this is not
      the issue I ignored it.
      Until I found out that the crash was on x264_pthread_create().
      It seems there is some problem with pthread once
      pthread_win32_process_detach_np() is called, it just won't work anymore.
    workaround: remove the call of pthread_win32_process_detach_np()
 At this point the encoder was working.
 4) crash on exit
    Once the x264 encoder was used, vlc will crash on exit, but I didn't look
    into that.

I can look closer at points 3 and 4 if you want me to.

regards
Domani Hannes