[x265] [PATCH] stats: remove --cu-stats option
Deepthi Nandakumar
deepthi at multicorewareinc.com
Fri May 29 07:20:35 CEST 2015
So, the general roadmap for stats collection is this:
x265 will keep all console outputs as-is, but disable cu-stats option. The
cu-stats will be replaced by a more powerful per-frame CU stats in
log-level 5. Now, the proposed log-levels will look like:
log-level 0(error) : only errors
log-level 1(warning) : only errors and warnings
log-level 2(summary) : errors, warnings, summary
log-level 3(info) : full x265 info, and above - default
log-level 4 (frame) : Basic frame-level info and above
log-level 5 (full) : Full frame-level info, lots of additional
statistics. This option is available only if csv is also enabled. Without
csv, this will default to log-level 4 outputs in console.
Comments appreciated.
Thanks,
Deepthi
On Thu, May 28, 2015 at 8:25 PM, Steve Borho <steve at borho.org> wrote:
> On 05/28, Divya Manivannan wrote:
> > # HG changeset patch
> > # User Divya Manivannan <divya at multicorewareinc.com>
> > # Date 1432810379 -19800
> > # Thu May 28 16:22:59 2015 +0530
> > # Node ID d0b3282c8621837e76d0a09cebe22fa663eb6c29
> > # Parent 68d8123c5fe7dcf543d9562d1a701c7d466e0a88
> > stats: remove --cu-stats option
> >
> > --cu-stats option collected data based on slicetype. This is removed and
> those data will be collected based on frame.
> > A warning message is displayed when --cu-stats option is given by the
> user.
>
> nit: CTRL+E in the commit editor will reflow paragraphs to reasonable
> text widths
>
> Before I would queue this, I would want to see a clear road-map for how
> the stats collection is going to be improved, because the end-of-encode
> logging by slicetype is a feature and this is removing it. x264 reports
> this data unconditionally.
>
> I want to see a road map on where all of this is going so we can discuss
> the particulars before we break any backward compatibility.
>
> If we're planning to remove frame-level log messages, then we should no
> longer base frame-level statistics gathering on --log-level, which is
> only supposed to regulate console logging.
>
> > diff -r 68d8123c5fe7 -r d0b3282c8621 doc/reST/cli.rst
> > --- a/doc/reST/cli.rst Wed May 27 14:16:34 2015 +0530
> > +++ b/doc/reST/cli.rst Thu May 28 16:22:59 2015 +0530
> > @@ -124,13 +124,6 @@
> > is more of a problem for P frames where some blocks are much more
> > expensive than others.
> >
> > -
> > -.. option:: --cu-stats, --no-cu-stats
> > -
> > - Records statistics on how each CU was coded (split depths and other
> > - mode decisions) and reports those statistics at the end of the
> > - encode. Default disabled
> > -
> > .. option:: --ssim, --no-ssim
> >
> > Calculate and report Structural Similarity values. It is
> > diff -r 68d8123c5fe7 -r d0b3282c8621 source/CMakeLists.txt
> > --- a/source/CMakeLists.txt Wed May 27 14:16:34 2015 +0530
> > +++ b/source/CMakeLists.txt Thu May 28 16:22:59 2015 +0530
> > @@ -30,7 +30,7 @@
> > mark_as_advanced(FPROFILE_USE FPROFILE_GENERATE NATIVE_BUILD)
> >
> > # X265_BUILD must be incremented each time the public API is changed
> > -set(X265_BUILD 60)
> > +set(X265_BUILD 61)
> > configure_file("${PROJECT_SOURCE_DIR}/x265.def.in"
> > "${PROJECT_BINARY_DIR}/x265.def")
> > configure_file("${PROJECT_SOURCE_DIR}/x265_config.h.in"
> > diff -r 68d8123c5fe7 -r d0b3282c8621 source/encoder/encoder.cpp
> > --- a/source/encoder/encoder.cpp Wed May 27 14:16:34 2015 +0530
> > +++ b/source/encoder/encoder.cpp Thu May 28 16:22:59 2015 +0530
> > @@ -1048,143 +1048,6 @@
> > #undef ELAPSED_SEC
> > #undef ELAPSED_MSEC
> > #endif
> > -
> > - if (!m_param->bLogCuStats)
> > - return;
> > -
> > - for (int sliceType = 2; sliceType >= 0; sliceType--)
> > - {
> > - if (sliceType == P_SLICE && !m_analyzeP.m_numPics)
> > - continue;
> > - if (sliceType == B_SLICE && !m_analyzeB.m_numPics)
> > - continue;
> > -
> > - StatisticLog finalLog;
> > - for (uint32_t depth = 0; depth <= g_maxCUDepth; depth++)
> > - {
> > - int cuSize = g_maxCUSize >> depth;
> > -
> > - for (int i = 0; i < m_param->frameNumThreads; i++)
> > - {
> > - StatisticLog& enclog =
> m_frameEncoder[i]->m_sliceTypeLog[sliceType];
> > - if (!depth)
> > - finalLog.totalCu += enclog.totalCu;
> > - finalLog.cntIntra[depth] += enclog.cntIntra[depth];
> > - for (int m = 0; m < INTER_MODES; m++)
> > - {
> > - if (m < INTRA_MODES)
> > - finalLog.cuIntraDistribution[depth][m] +=
> enclog.cuIntraDistribution[depth][m];
> > - finalLog.cuInterDistribution[depth][m] +=
> enclog.cuInterDistribution[depth][m];
> > - }
> > -
> > - if (cuSize == 8 && m_sps.quadtreeTULog2MinSize < 3)
> > - finalLog.cntIntraNxN += enclog.cntIntraNxN;
> > - if (sliceType != I_SLICE)
> > - {
> > - finalLog.cntTotalCu[depth] +=
> enclog.cntTotalCu[depth];
> > - finalLog.cntInter[depth] += enclog.cntInter[depth];
> > - finalLog.cntSkipCu[depth] +=
> enclog.cntSkipCu[depth];
> > - }
> > - }
> > -
> > - uint64_t cntInter, cntSkipCu, cntIntra = 0, cntIntraNxN =
> 0, encCu = 0;
> > - uint64_t cuInterDistribution[INTER_MODES],
> cuIntraDistribution[INTRA_MODES];
> > -
> > - // check for 0/0, if true assign 0 else calculate percentage
> > - for (int n = 0; n < INTER_MODES; n++)
> > - {
> > - if (!finalLog.cntInter[depth])
> > - cuInterDistribution[n] = 0;
> > - else
> > - cuInterDistribution[n] =
> (finalLog.cuInterDistribution[depth][n] * 100) / finalLog.cntInter[depth];
> > -
> > - if (n < INTRA_MODES)
> > - {
> > - if (!finalLog.cntIntra[depth])
> > - {
> > - cntIntraNxN = 0;
> > - cuIntraDistribution[n] = 0;
> > - }
> > - else
> > - {
> > - cntIntraNxN = (finalLog.cntIntraNxN * 100) /
> finalLog.cntIntra[depth];
> > - cuIntraDistribution[n] =
> (finalLog.cuIntraDistribution[depth][n] * 100) / finalLog.cntIntra[depth];
> > - }
> > - }
> > - }
> > -
> > - if (!finalLog.totalCu)
> > - encCu = 0;
> > - else if (sliceType == I_SLICE)
> > - {
> > - cntIntra = (finalLog.cntIntra[depth] * 100) /
> finalLog.totalCu;
> > - cntIntraNxN = (finalLog.cntIntraNxN * 100) /
> finalLog.totalCu;
> > - }
> > - else
> > - encCu = ((finalLog.cntIntra[depth] +
> finalLog.cntInter[depth]) * 100) / finalLog.totalCu;
> > -
> > - if (sliceType == I_SLICE)
> > - {
> > - cntInter = 0;
> > - cntSkipCu = 0;
> > - }
> > - else if (!finalLog.cntTotalCu[depth])
> > - {
> > - cntInter = 0;
> > - cntIntra = 0;
> > - cntSkipCu = 0;
> > - }
> > - else
> > - {
> > - cntInter = (finalLog.cntInter[depth] * 100) /
> finalLog.cntTotalCu[depth];
> > - cntIntra = (finalLog.cntIntra[depth] * 100) /
> finalLog.cntTotalCu[depth];
> > - cntSkipCu = (finalLog.cntSkipCu[depth] * 100) /
> finalLog.cntTotalCu[depth];
> > - }
> > -
> > - // print statistics
> > - char stats[256] = { 0 };
> > - int len = 0;
> > - if (sliceType != I_SLICE)
> > - len += sprintf(stats + len, " EncCU "X265_LL "%% Merge
> "X265_LL "%%", encCu, cntSkipCu);
> > -
> > - if (cntInter)
> > - {
> > - len += sprintf(stats + len, " Inter "X265_LL "%%",
> cntInter);
> > - if (m_param->bEnableAMP)
> > - len += sprintf(stats + len, "(%dx%d "X265_LL "%%
> %dx%d "X265_LL "%% %dx%d "X265_LL "%% AMP "X265_LL "%%)",
> > - cuSize, cuSize,
> cuInterDistribution[0],
> > - cuSize / 2, cuSize,
> cuInterDistribution[2],
> > - cuSize, cuSize / 2,
> cuInterDistribution[1],
> > - cuInterDistribution[3]);
> > - else if (m_param->bEnableRectInter)
> > - len += sprintf(stats + len, "(%dx%d "X265_LL "%%
> %dx%d "X265_LL "%% %dx%d "X265_LL "%%)",
> > - cuSize, cuSize,
> cuInterDistribution[0],
> > - cuSize / 2, cuSize,
> cuInterDistribution[2],
> > - cuSize, cuSize / 2,
> cuInterDistribution[1]);
> > - }
> > - if (cntIntra)
> > - {
> > - len += sprintf(stats + len, " Intra "X265_LL "%%(DC
> "X265_LL "%% P "X265_LL "%% Ang "X265_LL "%%",
> > - cntIntra, cuIntraDistribution[0],
> > - cuIntraDistribution[1],
> cuIntraDistribution[2]);
> > - if (sliceType != I_SLICE)
> > - {
> > - if (cuSize == 8 && m_sps.quadtreeTULog2MinSize < 3)
> > - len += sprintf(stats + len, " %dx%d "X265_LL
> "%%", cuSize / 2, cuSize / 2, cntIntraNxN);
> > - }
> > -
> > - len += sprintf(stats + len, ")");
> > - if (sliceType == I_SLICE)
> > - {
> > - if (cuSize == 8 && m_sps.quadtreeTULog2MinSize < 3)
> > - len += sprintf(stats + len, " %dx%d: "X265_LL
> "%%", cuSize / 2, cuSize / 2, cntIntraNxN);
> > - }
> > - }
> > - const char slicechars[] = "BPI";
> > - if (stats[0])
> > - x265_log(m_param, X265_LOG_INFO, "%c%-2d:%s\n",
> slicechars[sliceType], cuSize, stats);
> > - }
> > - }
> > }
> >
> > void Encoder::fetchStats(x265_stats *stats, size_t statsSizeBytes)
> > @@ -1902,6 +1765,9 @@
> > }
> > else
> > m_param->rc.qgSize = p->maxCUSize;
> > +
> > + if (p->bLogCuStats)
> > + x265_log(p, X265_LOG_WARNING, "--cu-stats option is replaced by
> --log-level 4 option\n");
> > }
> >
> > void Encoder::allocAnalysis(x265_analysis_data* analysis)
> > diff -r 68d8123c5fe7 -r d0b3282c8621 source/encoder/frameencoder.cpp
> > --- a/source/encoder/frameencoder.cpp Wed May 27 14:16:34 2015 +0530
> > +++ b/source/encoder/frameencoder.cpp Thu May 28 16:22:59 2015 +0530
> > @@ -904,7 +904,7 @@
> > // Completed CU processing
> > curRow.completed++;
> >
> > - if (m_param->bLogCuStats || m_param->rc.bStatWrite)
> > + if (m_param->rc.bStatWrite)
> > curEncData.m_rowStat[row].sumQpAq +=
> collectCTUStatistics(*ctu, qTreeInterCnt, qTreeIntraCnt, qTreeSkipCnt);
> > else if (m_param->rc.aqMode)
> > curEncData.m_rowStat[row].sumQpAq += calcCTUQP(*ctu);
> > diff -r 68d8123c5fe7 -r d0b3282c8621 source/x265cli.h
> > --- a/source/x265cli.h Wed May 27 14:16:34 2015 +0530
> > +++ b/source/x265cli.h Thu May 28 16:22:59 2015 +0530
> > @@ -247,7 +247,6 @@
> > H0("-D/--output-depth 8|10 Output bit depth (also
> internal bit depth). Default %d\n", param->internalBitDepth);
> > H0(" --log-level <string> Logging level: none error
> warning info debug full. Default %s\n", x265::logLevelNames[param->logLevel
> + 1]);
> > H0(" --no-progress Disable CLI progress
> reports\n");
> > - H0(" --[no-]cu-stats Enable logging stats about
> distribution of cu across all modes. Default %s\n",OPT(param->bLogCuStats));
> > H1(" --csv <filename> Comma separated log file, log
> level >= 3 frame log, else one line per run\n");
> > H0("\nInput Options:\n");
> > H0(" --input <filename> Raw YUV or Y4M input file
> name. `-` for stdin\n");
> > _______________________________________________
> > x265-devel mailing list
> > x265-devel at videolan.org
> > https://mailman.videolan.org/listinfo/x265-devel
>
> --
> Steve Borho
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20150529/db51e8e1/attachment-0001.html>
More information about the x265-devel
mailing list