[x265] [PATCH x265] dynamic-refine: restart training at the beginning of each GOP

[Ashok Kumar Mishra]

On Mon, Apr 2, 2018 at 4:21 PM, Bhavna Hariharan <
bhavna at multicorewareinc.com> wrote:

>
> On Mon, Apr 2, 2018 at 4:04 PM, Ashok Kumar Mishra <
> ashok at multicorewareinc.com> wrote:
>
>>
>>
>> On Fri, Mar 30, 2018 at 5:26 PM, <bhavna at multicorewareinc.com> wrote:
>>
>>> # HG changeset patch
>>> # User Bhavna Hariharan <bhavna at multicorewareinc.com>
>>> # Date 1522408210 -19800
>>> #      Fri Mar 30 16:40:10 2018 +0530
>>> # Node ID ae72210ad6e846062572ef8a02970b74052c2f1c
>>> # Parent  3440a56acc7865dcdea725b8ce7755450209a8ee
>>> dynamic-refine: restart training at the beginning of each GOP
>>>
>>> Earlier, the training restarted at scenecuts. Now, the algorithm
>>> restarts at
>>> the beginning of each GOP in addition to the scenecuts.
>>>
>>> diff -r 3440a56acc78 -r ae72210ad6e8 source/encoder/encoder.cpp
>>> --- a/source/encoder/encoder.cpp        Thu Mar 29 14:35:50 2018 +0530
>>> +++ b/source/encoder/encoder.cpp        Fri Mar 30 16:40:10 2018 +0530
>>> @@ -416,7 +416,8 @@
>>>
>>>      if (m_param->bDynamicRefine)
>>>      {
>>> -        int size = m_param->totalFrames * m_param->maxCUDepth *
>>> X265_REFINE_INTER_LEVELS;
>>> +        /* Allocate memory for 1 GOP and reuse it for the subsequent
>>> GOPs */
>>> +        int size = (m_param->keyframeMax + m_param->lookaheadDepth) *
>>> m_param->maxCUDepth * X265_REFINE_INTER_LEVELS;
>>>          CHECKED_MALLOC_ZERO(m_variance, uint64_t, size);
>>>          CHECKED_MALLOC_ZERO(m_rdCost, uint64_t, size);
>>>          CHECKED_MALLOC_ZERO(m_trainingCount, uint32_t, size);
>>> diff -r 3440a56acc78 -r ae72210ad6e8 source/encoder/frameencoder.cpp
>>> --- a/source/encoder/frameencoder.cpp   Thu Mar 29 14:35:50 2018 +0530
>>> +++ b/source/encoder/frameencoder.cpp   Fri Mar 30 16:40:10 2018 +0530
>>> @@ -1463,6 +1463,7 @@
>>>
>>>          if (m_param->bDynamicRefine)
>>>          {
>>> +            if (m_top->m_startPoint <= m_frame->m_encodeOrder) // Avoid
>>> collecting data that will not be used by future frames.
>>>              {
>>>                  ScopedLock dynLock(m_top->m_dynamicRefineLock);
>>>                  for (uint32_t i = 0; i < X265_REFINE_INTER_LEVELS; i++)
>>> @@ -1470,7 +1471,8 @@
>>>                      for (uint32_t depth = 0; depth <
>>> m_param->maxCUDepth; depth++)
>>>                      {
>>>                          int offset = (depth * X265_REFINE_INTER_LEVELS)
>>> + i;
>>> -                        int index = (m_frame->m_encodeOrder *
>>> X265_REFINE_INTER_LEVELS * m_param->maxCUDepth) + offset;
>>> +                        int curFrameIndex = m_frame->m_encodeOrder -
>>> m_top->m_startPoint;
>>> +                        int index = (curFrameIndex *
>>> X265_REFINE_INTER_LEVELS * m_param->maxCUDepth) + offset;
>>>                          if (ctu->m_collectCUCount[offset])
>>>                          {
>>>                              m_top->m_variance[index] +=
>>> ctu->m_collectCUVariance[offset];
>>> @@ -1869,8 +1871,14 @@
>>>
>>>  void FrameEncoder::computeAvgTrainingData()
>>>  {
>>> -    if (m_frame->m_lowres.bScenecut)
>>> +    if (m_frame->m_lowres.bScenecut || m_frame->m_lowres.bKeyframe)
>>> +    {
>>>          m_top->m_startPoint = m_frame->m_encodeOrder;
>>> +        int size = (m_param->keyframeMax + m_param->lookaheadDepth) *
>>> m_param->maxCUDepth * X265_REFINE_INTER_LEVELS;
>>> +        memset(m_top->m_variance, 0, size * sizeof(uint64_t));
>>> +        memset(m_top->m_rdCost, 0, size * sizeof(uint64_t));
>>> +        memset(m_top->m_trainingCount, 0, size * sizeof(uint32_t));
>>>
>>
>> Is it required to reset the buffer every time? I believe you might be
>> over writing those buffers periodically. Correct me if i am wrong.
>>
>
> We will be incrementally adding to the buffer so the reset is required
> when we want to restart the training.
>
>>
>> +    }
>>>
>>>      if (m_frame->m_encodeOrder - m_top->m_startPoint < 2 *
>>> m_param->frameNumThreads)
>>>          m_frame->m_classifyFrame = false;
>>> @@ -1884,8 +1892,8 @@
>>>
>>>      if (m_frame->m_classifyFrame)
>>>      {
>>> -        uint32_t limit = m_frame->m_encodeOrder -
>>> m_param->frameNumThreads - 1;
>>> -        for (uint32_t i = m_top->m_startPoint + 1; i < limit; i++)
>>> +        uint32_t limit = m_frame->m_encodeOrder - m_top->m_startPoint -
>>> m_param->frameNumThreads;
>>> +        for (uint32_t i = 1; i < limit; i++)
>>>          {
>>>              for (uint32_t j = 0; j < X265_REFINE_INTER_LEVELS; j++)
>>>              {
>>>
>>> _______________________________________________
>>> x265-devel mailing list
>>> x265-devel at videolan.org
>>> https://mailman.videolan.org/listinfo/x265-devel
>>>
>>>
>>
>> _______________________________________________
>> x265-devel mailing list
>> x265-devel at videolan.org
>> https://mailman.videolan.org/listinfo/x265-devel
>>
>>
>
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
>
Pushed.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20180402/2bcd79e0/attachment-0001.html>