[Pixman] [BUG] SIGSEGV in sse2_fill

[Frédéric Fauberteau]

I encounter a xorg segfault after building it with the following
optimization CFLAGS: -march=athlon64-sse3 -mfpmath=sse

I noticed that height goes from 1050 in _pixman_implementation_fill() to
1041 in sse2_fill() ...?

Reading symbols from /usr/pkg/bin/X...done.
(gdb) run :0
Starting program: /usr/pkg/bin/X :0

X.Org X Server 1.20.0
X Protocol Version 11, Revision 0
Build Operating System: NetBSD-8.0-x86_64 The NetBSD Foundation
Current Operating System: NetBSD hydralisk 8.0 NetBSD 8.0 (HYDRALISK)
#2: Thu Aug 23 13:57:31 CEST 2018
***@hydralisk:/usr/obj/sys/arch/amd64/compile/HYDRALISK amd64
Build Date: 29 August 2018 10:12:23AM

Current version of pixman: 0.34.0
Before reporting problems, check http://wiki.x.org
to make sure that you have the latest version.
Markers: (--) probed, (**) from config file, (==) default setting,
(++) from command line, (!!) notice, (II) informational,
(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(==) Log file: "/var/log/Xorg.0.log", Time: Wed Aug 29 10:19:40 2018
(==) Using config file: "/etc/xorg.conf"
(==) Using system config directory "/usr/pkg/share/X11/xorg.conf.d"
(II) [KMS] Kernel modesetting enabled.

Program received signal SIGSEGV, Segmentation fault.
sse2_fill (imp=<optimized out>, bits=<optimized out>, stride=<optimized
out>, bpp=<optimized out>, x=<optimized out>, y=<optimized out>,
width=2960, height=1041, filler=0) at pixman-sse2.c:3408
3408 pixman-sse2.c: No such file or directory.
(gdb) bt
#0 sse2_fill (imp=<optimized out>, bits=<optimized out>,
stride=<optimized out>, bpp=<optimized out>, x=<optimized out>,
y=<optimized out>, width=2960, height=1041, filler=0) at
pixman-sse2.c:3408
#1 0x000075dc0f8509ef in _pixman_implementation_fill
(imp=0x75dc10304000, bits=***@entry=0x75dc0ffe6aa0,
stride=***@entry=2976, bpp=***@entry=32, x=***@entry=0, y=***@entry=0,
width=***@entry=2960, height=***@entry=1050, filler=<optimized
out>, ***@entry=0)
at pixman-implementation.c:277
#2 0x000075dc0f80ba14 in pixman_fill (bits=***@entry=0x75dc0ffe6aa0,
stride=***@entry=2976, bpp=***@entry=32, x=***@entry=0, y=***@entry=0,
width=***@entry=2960, height=***@entry=1050, filler=0) at
pixman.c:834
#3 0x000075dc06c0de9c in fbFill
(pDrawable=***@entry=0x75dc0fb447c0, pGC=***@entry=0x75dc0ff88a00,
x=***@entry=0, y=***@entry=0, width=***@entry=2960,
height=***@entry=1050) at fbfill.c:125
#4 0x000075dc06c0e720 in fbPolyFillRect
(pDrawable=***@entry=0x75dc0fb447c0, pGC=***@entry=0x75dc0ff88a00,
nrect=<optimized out>, ***@entry=1, prect=0x75dc0e33b088,
***@entry=0x75dc0e33b080) at fbfillrect.c:72
#5 0x000075dc06819c23 in glamor_poly_fill_rect_bail
(prect=0x75dc0e33b080, nrect=1, gc=0x75dc0ff88a00,
drawable=0x75dc0fb447c0) at glamor_rects.c:174
#6 glamor_poly_fill_rect (drawable=0x75dc0fb447c0, gc=0x75dc0ff88a00,
nrect=1, prect=0x75dc0e33b080) at glamor_rects.c:186
#7 0x0000000000503fd8 in damagePolyFillRect (pDrawable=0x75dc0fb447c0,
pGC=0x75dc0ff88a00, nRects=1, pRects=<optimized out>) at damage.c:1204
#8 0x000000000055dfa6 in miPaintWindow (pWin=<optimized out>,
prgn=0x7f7fff7d3de0, what=<optimized out>) at miexpose.c:540
#9 0x000000000055dc86 in miWindowExposures (pWin=0x75dc0fb447c0,
prgn=0x7f7fff7d3de0) at miexpose.c:394
#10 0x000000000048008e in xf86XVWindowExposures (pWin=0x75dc0fb447c0,
reg1=0x7f7fff7d3de0) at xf86xv.c:1062
#11 0x000075dc07c480b0 in RADEONWindowExposures_oneshot
(pWin=0x75dc0fb447c0, pRegion=0x7f7fff7d3de0) at radeon_kms.c:1681
#12 0x000000000045b7fd in MapWindow (pWin=0x75dc0fb447c0,
client=<optimized out>) at window.c:2722
#13 0x00000000004341de in dix_main (argc=2, argv=0x7f7fff7d3ec0,
envp=<optimized out>) at main.c:247
#14 0x000000000041ebdb in ___start ()
#15 0x00007f7fa7603382 in _rtld () from /usr/libexec/ld.elf_so
#16 0x00007f7fff7d4571 in ?? ()
#17 0x00007f7fff7d457f in ?? ()
#18 0x0000000000000000 in ?? ()

[Adam Jackson]

Post by Frédéric Fauberteau
I encounter a xorg segfault after building it with the following
optimization CFLAGS: -march=athlon64-sse3 -mfpmath=sse

This is almost certainly a bug in either the radeon driver or in the X
server. pixman is usually innocent in backtraces like this, as it just
draws where it's told to.

Post by Frédéric Fauberteau
I noticed that height goes from 1050 in _pixman_implementation_fill() to
1041 in sse2_fill() ...?

It happened to fill nine rows of pixels before segfaulting. There's
probably no deep or numerological reason for this, other than: the
place where xserver told pixman to start drawing, happened to be about
that far away from unmapped memory.

This is a curious backtrace though. You're crashing while trying to
draw the black solid fill for the initial map of the root window. Fine,
but you're doing so in software, even though you have glamor enabled,
and glamor surely can usually accelerate solid fills. So you're hitting
a software fallback for some reason, and if I had to guess...

Post by Frédéric Fauberteau
Program received signal SIGSEGV, Segmentation fault.
sse2_fill (imp=<optimized out>, bits=<optimized out>, stride=<optimized
out>, bpp=<optimized out>, x=<optimized out>, y=<optimized out>,
width=2960, height=1041, filler=0) at pixman-sse2.c:3408

... that width number is where I'd start guessing. You don't say what
radeon this is, but for some old ones (R300-ish if memory serves, I
don't remember where in Mesa to look it up) that's wider than the 3D
engine can draw to. That would explain the fallback, and it would
suggest that even if the crash were fixed the interactive experience
would be miserably slow.

- ajax

[Frédéric Fauberteau]

Post by Adam Jackson

Post by Frédéric Fauberteau
I encounter a xorg segfault after building it with the following
optimization CFLAGS: -march=athlon64-sse3 -mfpmath=sse

This is almost certainly a bug in either the radeon driver or in the X
server. pixman is usually innocent in backtraces like this, as it just
draws where it's told to.

Sure, it makes sense. Sorry, pixman was the last suspect on top of
backtrace stack ;)

Post by Adam Jackson

Post by Frédéric Fauberteau
I noticed that height goes from 1050 in _pixman_implementation_fill() to
1041 in sse2_fill() ...?

It happened to fill nine rows of pixels before segfaulting. There's
probably no deep or numerological reason for this, other than: the
place where xserver told pixman to start drawing, happened to be about
that far away from unmapped memory.
This is a curious backtrace though. You're crashing while trying to
draw the black solid fill for the initial map of the root window. Fine,
but you're doing so in software, even though you have glamor enabled,
and glamor surely can usually accelerate solid fills. So you're hitting
a software fallback for some reason, and if I had to guess...

The area to fill is 2960x1050 but actually, I have two screens:
----------------- -------------
| | | |
| | | |
| 1680x1050 | | 1280x1024 |
| | | |
| | |-----------|
-----------------

Do you think it could be a reason to write in an unmapped region...?

Post by Adam Jackson

Post by Frédéric Fauberteau
Program received signal SIGSEGV, Segmentation fault.
sse2_fill (imp=<optimized out>, bits=<optimized out>,
stride=<optimized
out>, bpp=<optimized out>, x=<optimized out>, y=<optimized out>,
width=2960, height=1041, filler=0) at pixman-sse2.c:3408

... that width number is where I'd start guessing. You don't say what
radeon this is, but for some old ones (R300-ish if memory serves, I
don't remember where in Mesa to look it up) that's wider than the 3D
engine can draw to. That would explain the fallback, and it would
suggest that even if the crash were fixed the interactive experience
would be miserably slow.

It is a RS780/RS880 (Radeon HD 3200). If the bug comes from the Mesa
driver, it's a big issue since we are totally late with the update (we
are on MesaLib 11.2.2)

Post by Adam Jackson
- ajax

Fred