Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(480121)

Issue 932: Fix typo in ARM DCT. There are still overflows.

Can't Edit
Can't Publish+Mail
Start Review
Created:
3 years, 3 months ago by tdaede
Modified:
3 years, 2 months ago
Reviewers:
negge
Visibility:
Public.

Patch Set 1 #

Unified diffs Side-by-side diffs Delta from patch set Stats Patch
M src/arm/armdct.c View 1 chunk +1 line, -1 line 0 comments Download

Messages

Total messages: 2
tdaede
3 years, 3 months ago #1
negge
3 years, 2 months ago #2
I was able to apply this patch with modified 934.  I don't know if this change
makes it any better, but it still fails src/tests/dcttest

Testing NEON DCT...
1-D ranges:
 Min:  -512  -472  -511  -473
 Max:   510   472   511   471
2-D scaled, prefiltered ranges:
 Min  0:  -31449  -31500  -28077  -26564
 Max  0:   31444   31500   28081   26564
 Min  1:  -31500  -31553  -28127  -26610
 Max  1:   31500   31553   28127   26606
 Min  2:  -28079  -28127  -25072  -23719
 Max  2:   28078   28127   25072   23721
 Min  3:  -26563  -26609  -23719  -22440
 Max  3:   26563   26609   23719   22440
od_bin_idct4 basis:
 0.50000  0.65234  0.50000  0.26953
 0.50000  0.26953 -0.50000 -0.65234
 0.50000 -0.26953 -0.50000  0.65234
 0.50000 -0.65234  0.50000 -0.27344
Scaled type-II iDCT basis:
 0.50000  0.65328  0.50000  0.27060
 0.50000  0.27060 -0.50000 -0.65328
 0.50000 -0.27060 -0.50000  0.65328
 0.50000 -0.65328  0.50000 -0.27060

od_bin_fdct4 basis:
 0.50000  0.50000  0.50000  0.50000
 0.65234  0.26953 -0.26953 -0.65234
 0.50000 -0.50000 -0.50000  0.50000
 0.27344 -0.65234  0.65234 -0.27344
Scaled type-II DCT basis:
 0.50000  0.50000  0.50000  0.50000
 0.65328  0.27060 -0.27060 -0.65328
 0.50000 -0.50000 -0.50000  0.50000
 0.27060 -0.65328  0.65328 -0.27060
MSE: 8.6209183358465040095477711218352e-07

Mismatch:
in:
        112 -2672 -1568 272
        3664 -2704 1648 -2256
        -48 -3088 -3424 -912
        -1840 -1088 3952 288
xform:
        -1242 -2116 1254 -1235
        -1805 6 1733 2948
        -685 -514 5118 -2442
        694 2592 -423 -4030
out:
        -1306 -13 2083 2274
        729 -1745 66 -2320
        -3340 4636 934 3823
        -1892 -2137 -1369 -1493

Mismatch:
in:
        2176 1184 2176 2288
        2640 -2864 1024 -1520
        -1264 3408 160 -816
        864 2336 3520 3024
xform:
        1852 1955 -2094 2661
        1807 2708 963 -188
        3076 -933 1182 -3204
        3264 2589 -280 -2565
out:
        1859 -498 -1619 3290
        3280 1011 1551 -2751
        1777 4288 -109 1520
        764 1722 2699 1640

Mismatch:
in:
        2992 1424 2112 656
        -912 -1184 -2464 2672
        -704 -304 3920 -3072
        -2368 3744 1936 -752
xform:
        1517 2992 -3768 -1924
        1682 1147 3543 -3222
        387 1236 1389 -820
        550 1947 -814 3920
out:
        -104 469 -3283 2467
        2690 2728 -2474 2067
        1197 93 2996 -346
        3946 316 2473 -2478
Sign in to reply to this message.

RSS Feeds Recent Issues | This issue
This is Rietveld