BZ #79: [arm] mfloat-abi=softfp: the runtime fails to run anything

Status fields:

creation_ts:2008-06-12 11:35
component:vm
version:default branch
rep_platform:arm
op_sys:Linux
bug_status:RESOLVED
resolution:WONTFIX
reporter:thebohemian@gmx.net
When the CFLAGS for cacao on ARM contain -mfloat-abi=softfp the runtime crashes
immediately.

I am adding more infos about this soon.

Comment #1 by twisti@complang.tuwien.ac.at on 2008-09-16 10:51:47

Is there still "more info coming soon"? ;-)

Comment #2 by thebohemian@gmx.net on 2008-09-17 22:48:31

Ups, sorry. Forgot about this one.

The bug originally targets Cacao binaries form ARM that do not have these patches
applied:

http://mips.complang.tuwien.ac.at/hg/cacao/rev/d88bf99453b6
http://mips.complang.tuwien.ac.at/hg/cacao/rev/a67fe1cb2b81

With those two it works mostly nicely. Mostly because I am seeing an occassional
'illegal instruction' now.

My current 'test case' is: Run midpath with SWT backend on the beagleboard. That is
rather big. I try to find a smaller way to reproduce the problem.

Comment #3 by twisti@complang.tuwien.ac.at on 2008-09-18 11:24:43

Interesting that -mfloat-abi=softfp is actually VFP... is it?

Comment #4 by thebohemian@gmx.net on 2008-09-20 16:44:30

> Interesting that -mfloat-abi=softfp is actually VFP... is it?
I think so.

After reading about bug #85 I wonder if this might be the problem I am seeing. I will
test the patch committed for this.

Comment #5 by thebohemian@gmx.net on 2008-09-24 17:22:38

I applied the patch from bug #85 for arm which changes the sigill handler but the
problem persists:

It looks like this btw:

LOG: [0x40183000] md_signal_handler_sigill: Unknown illegal instruction 0xed2d8102 at
0x4113da64
LOG: [0x40183000] Aborting...

Comment #6 by twisti@complang.tuwien.ac.at on 2008-09-24 17:40:27

Please disassemble the illegal instruction manually or compile CACAO with --enable-
disassembler.

Comment #7 by thebohemian@gmx.net on 2008-09-30 15:52:32

Cacao compiled with --enable-dissassembler outputs this:

LOG: [0x40246210] md_signal_handler_sigill: Unknown illegal instruction 0xed2d8102 at
0x41963d54
0x41963d54:   ed2d8102    stfd  f0, [sp, #-8]!

Comment #8 by twisti@complang.tuwien.ac.at on 2008-09-30 15:59:41

There you are.  Your board does not support floating point instructions.

Comment #9 by thebohemian@gmx.net on 2008-09-30 16:03:20

Since I found these comments around the code that generates the stfd instruction:

/* TODO: this is only a hack, since we use R0/R1 for float return! */
        /* this depends on gcc; it is independent from our ENABLE_SOFTFLOAT define */

I think it is important to note that I am using GCC 4.3.1 to compile cacao. The exact
ABI flags are:

"-march=armv7-a -mtune=cortex-a8 -mfpu=neon -mfloat-abi=softfp"

Comment #10 by twisti@complang.tuwien.ac.at on 2008-09-30 16:14:45

Michi, do you know something about that comment?

Comment #11 by thebohemian@gmx.net on 2008-10-01 14:50:10

Created an attachment (id=47)
demo code

Comment #12 by michi@complang.tuwien.ac.at on 2008-10-01 14:54:22

These instructions are needed because of a slight difference between two ABIs.

Cacao ABI: Always use R0/R1 for float or double return values. There is no difference
between soft FP or hardware-accelerated FP.

GCC ABI (as far as I understood it): In case of soft FP the R0/R1 register pair is used
as well. No problems there. But for hard FP the register F0 (iirc) is used for return
values. So we need to copy this register after builtin or native functions return to our
JIT code.

A testcase for the situation described above would be a native method returning a float
or double value.

It is possible that the "neon" FPU (so far never heard of it, sorry) needs another
instruction here. I'll look into it in the following days.

Is the "neon" FPU some kind of VFP?
Can you post the assembler output of GCC for a simple C function returning a double (or
float) value?

Comment #13 by thebohemian@gmx.net on 2008-10-01 14:58:22

In order to examine this situation in detail I wrote a small demo app. The commented
stuff isn't that important except that I got those examples working on the ARM hw I have
here. So single and double precision VFP instructions are working.

The last example is the interesting one. If you compile it with:
"-mfloat-abi=softfp -mfpu=vfp" the assembler will tell you that ldfd/stfd are not
supported.

If you compile it with "-mfloat-abi=softfp -mfpu=fpa" it will mourn about the fldd/fsdd
instructions.

So in my opinion cacao should not generate code containing those FPA instruction when
the target system uses a VFP.

Comment #14 by thebohemian@gmx.net on 2008-10-01 15:17:58

(In reply to comment #12)
> GCC ABI (as far as I understood it): In case of soft FP the R0/R1 register pair
> is used as well. No problems there. But for hard FP the register F0 (iirc) is
> used for return values. So we need to copy this register after builtin or
> native functions return to our JIT code.
GCC actually supports three fp-related calling conventions:
soft - emulation I suppose
hard - real FPU, actually so called 'FPA' hardware
softfp - real FPU, to be used with FPA and VFP hardware (__SOFTFP__ is not defined in
this mode)

> Is the "neon" FPU some kind of VFP?
The Neon unit is just another VFP, it can be part of the armv7 CPUs. There seems to be
one crucial distinction to the earlier VFPs: It does not support operations like
stfd/ldfd (= illegal instruction).

A binary containing those instructions runs fine on a armv6 /w VFP.

> Can you post the assembler output of GCC for a simple C function returning a
> double (or float) value?
Will do so.

How can I modify the codegenerator to no emit stfd/ldfd any more and instead generate
the functional equivalent fldd/fstd instructions? (They are described on page 908
(C3.2.1) in issue "I" of the ARM ARM.)

Comment #15 by michi@complang.tuwien.ac.at on 2008-10-01 15:20:11

Created an attachment (id=48)
Sample C functions returning float and double

Can you please post the assembler code that your GCC generates for the attached file.
You can do this with ...

$ gcc [your-fp-options] float_return.c -S -o float_return.S

And a list of all the defines that your GCC generates could be helpful as well

$ touch tmp.c
$ gcc [your-fp-options] -E -dM tmp.c

Thanks in advance!

Comment #16 by michi@complang.tuwien.ac.at on 2008-10-01 15:24:08

> > Is the "neon" FPU some kind of VFP?
> The Neon unit is just another VFP, it can be part of the armv7 CPUs. There
> seems to be one crucial distinction to the earlier VFPs: It does not support
> operations like stfd/ldfd (= illegal instruction).
>
> A binary containing those instructions runs fine on a armv6 /w VFP.

That is interesting! Didn't know that, thanks for the clarification!

> > Can you post the assembler output of GCC for a simple C function returning a
> > double (or float) value?
> Will do so.

Perfect.

> How can I modify the codegenerator to no emit stfd/ldfd any more and instead
> generate the functional equivalent fldd/fstd instructions? (They are described
> on page 908 (C3.2.1) in issue "I" of the ARM ARM.)

Generally it does not generate these instructions. There are only two places still
generating the code, both marked with the comment you already mentioned above. The
instructions are hardcoded as instructions words using the "DCD()" macro in codegen.c.
Maybe I'll find a cleaner way to do this, so we don't need to call it a "hack"
anylonger.

Comment #17 by thebohemian@gmx.net on 2008-10-01 15:26:29

Created an attachment (id=49)
demo code for fp-related calling conventions

Comment #18 by thebohemian@gmx.net on 2008-10-01 15:31:17

Created an attachment (id=50)
calling code assembler output

I've compiled the code with various -mfpu and -mfloat-abi options.

Comment #19 by thebohemian@gmx.net on 2008-10-01 15:37:30

Created an attachment (id=51)
calling code assembler output & defines

This time with neon options and all the defines.

Comment #20 by thebohemian@gmx.net on 2008-10-01 16:06:36

More info:
Neon does not support stfs/ldfs either.

FLDS/FSTS are functional equaivalent but they lack the writeback mechanism.

I am trying a workaround for the two hacks in arm/codegen.c that makes use of the the
FSTD instruction and an explicit "sub sp, sp, #8".

Comment #21 by thebohemian@gmx.net on 2008-10-01 16:09:14

> I am trying a workaround for the two hacks in arm/codegen.c that makes use of
> the the FSTD instruction and an explicit "sub sp, sp, #8".
Yay, that worked!

Comment #22 by michi@complang.tuwien.ac.at on 2008-10-01 16:19:14

Ok, first of all the good news:

Most probably your configuration should work without the two code segments inside the
"#if !defined(__SOFTFP__)". Try to disable them completely and see if it still works. I
fear we do not have a regression test for this, but I'll write one. There seems to be
exactly one configuration which needs this hack, and that is with -mfloat-abi=hard
-mfpu=fpa.

Now, to the bad news:

There is no GCC define which distinguishes this configuration from others. If you run a
diff on the generated defines you'll see what I am talking about. This is a real problem
for us, since we highly depend on the GCC ABI but have no way of finding out which one
it is using.

Have to think about this a little longer ...

But I'll definitely write a regression test for this!

Comment #23 by thebohemian@gmx.net on 2008-10-01 16:47:20

Created an attachment (id=53)
disable fp-return value hack for non-fpa configuration

AFAIU the hack should not be available for non-fpa targets. So depending on the non-
existance of __VFP_FP__ should work, right?

I fixed the issue with the above patch.

Comment #24 by michi@complang.tuwien.ac.at on 2008-10-01 16:55:36

> AFAIU the hack should not be available for non-fpa targets. So depending on the
> non-existance of __VFP_FP__ should work, right?

Only partly correct. If you take one configuration with "-mfloat-abi=hard -mfpu=fpa" and
another one with "-mfloat-abi=softfp -mfpu=fpa", then you'll get exactly the same
defines, but two different ABIs as return values are concerned. And these are exactly
the two configurations we cannot distinguish.

Comment #25 by thebohemian@gmx.net on 2008-10-01 17:29:50

(In reply to comment #24)
> Only partly correct. If you take one configuration with "-mfloat-abi=hard
> -mfpu=fpa" and another one with "-mfloat-abi=softfp -mfpu=fpa", then you'll get
> exactly the same defines, but two different ABIs as return values are
> concerned. And these are exactly the two configurations we cannot distinguish.
Ok, I see.

Looks like an explicit configure option is the only way to go here.

Comment #26 by stefan@complang.tuwien.ac.at on 2010-09-22 22:50:24

Works with soft or hard, no need for softfp.

Comment #27 by stefan@complang.tuwien.ac.at on 2010-09-25 18:59:37

In fact, it should just work, using --enable-softfloat.

Attachment id=47

date:2008-10-01 14:50
desc:demo code
type:text/plain
download:asmtest.c

Attachment id=48

date:2008-10-01 15:20
desc:Sample C functions returning float and double
type:text/plain
download:float_return.c

Attachment id=49

date:2008-10-01 15:26
desc:demo code for fp-related calling conventions
type:text/plain
download:callingtest.c

Attachment id=50

date:2008-10-01 15:31
desc:calling code assembler output
type:application/x-bzip
download:callingtest.tar.bz2

Attachment id=51

date:2008-10-01 15:37
desc:calling code assembler output & defines
type:application/x-bzip
download:callingtest.tar.bz2

Attachment id=53

date:2008-10-01 16:47
desc:disable fp-return value hack for non-fpa configuration
type:text/plain
download:neon-compat.diff