Page Menu
Home
c4science
Search
Configure Global Search
Log In
Files
F61129990
AssemblyTests.md
No One
Temporary
Actions
Download File
Edit File
Delete File
View Transforms
Subscribe
Mute Notifications
Award Token
Subscribers
None
File Metadata
Details
File Info
Storage
Attached
Created
Sat, May 4, 17:43
Size
5 KB
Mime Type
text/x-c
Expires
Mon, May 6, 17:43 (2 d)
Engine
blob
Format
Raw Data
Handle
17469598
Attached To
R12667 gbench-stream
AssemblyTests.md
View Options
#
Assembly
Tests
The
Benchmark
library
provides
a
number
of
functions
whose
primary
purpose
in
to
affect
assembly
generation
,
including
`DoNotOptimize`
and
`ClobberMemory`
.
In
addition
there
are
other
functions
,
such
as
`KeepRunning`
,
for
which
generating
good
assembly
is
paramount
.
For
these
functions
it
'
s
important
to
have
tests
that
verify
the
correctness
and
quality
of
the
implementation
.
This
requires
testing
the
code
generated
by
the
compiler
.
This
document
describes
how
the
Benchmark
library
tests
compiler
output
,
as
well
as
how
to
properly
write
new
tests
.
##
Anatomy
of
a
Test
Writing
a
test
has
two
steps
:
*
Write
the
code
you
want
to
generate
assembly
for
.
*
Add
`// CHECK`
lines
to
match
against
the
verified
assembly
.
Example
:
```
c
++
// CHECK-LABEL: test_add:
extern
"C"
int
test_add
()
{
extern
int
ExternInt
;
return
ExternInt
+
1
;
// CHECK: movl ExternInt(%rip), %eax
// CHECK: addl %eax
// CHECK: ret
}
```
####
LLVM
Filecheck
[
LLVM
'
s
Filecheck
](
https
:
//llvm.org/docs/CommandGuide/FileCheck.html)
is
used
to
test
the
generated
assembly
against
the
`// CHECK`
lines
specified
in
the
tests
source
file
.
Please
see
the
documentation
linked
above
for
information
on
how
to
write
`CHECK`
directives
.
####
Tips
and
Tricks
:
*
Tests
should
match
the
minimal
amount
of
output
required
to
establish
correctness
.
`CHECK`
directives
don
'
t
have
to
match
on
the
exact
next
line
after
the
previous
match
,
so
tests
should
omit
checks
for
unimportant
bits
of
assembly
.
([
`CHECK-NEXT`
](
https
:
//llvm.org/docs/CommandGuide/FileCheck.html#the-check-next-directive)
can
be
used
to
ensure
a
match
occurs
exactly
after
the
previous
match
).
*
The
tests
are
compiled
with
`-O3 -g0`
.
So
we
'
re
only
testing
the
optimized
output
.
*
The
assembly
output
is
further
cleaned
up
using
`tools/strip_asm.py`
.
This
removes
comments
,
assembler
directives
,
and
unused
labels
before
the
test
is
run
.
*
The
generated
and
stripped
assembly
file
for
a
test
is
output
under
`<build-directory>/test/<test-name>.s`
*
Filecheck
supports
using
[
`CHECK`
prefixes
](
https
:
//llvm.org/docs/CommandGuide/FileCheck.html#cmdoption-check-prefixes)
to
specify
lines
that
should
only
match
in
certain
situations
.
The
Benchmark
tests
use
`CHECK-CLANG` and `CHECK-GNU`
for
lines
that
are
only
expected
to
match
Clang
or
GCC
'
s
output
respectively
.
Normal
`CHECK` lines match against all compilers. (Note: `CHECK-NOT`
and
`CHECK-LABEL`
are
NOT
prefixes
.
They
are
versions
of
non
-
prefixed
`CHECK`
lines
)
*
Use
`extern "C"`
to
disable
name
mangling
for
specific
functions
.
This
makes
them
easier
to
name
in
the
`CHECK`
lines
.
##
Problems
Writing
Portable
Tests
Writing
tests
which
check
the
code
generated
by
a
compiler
are
inherently
non
-
portable
.
Different
compilers
and
even
different
compiler
versions
may
generate
entirely
different
code
.
The
Benchmark
tests
must
tolerate
this
.
LLVM
Filecheck
provides
a
number
of
mechanisms
to
help
write
"more portable"
tests
;
including
[
matching
using
regular
expressions
](
https
:
//llvm.org/docs/CommandGuide/FileCheck.html#filecheck-pattern-matching-syntax),
allowing
the
creation
of
[
named
variables
](
https
:
//llvm.org/docs/CommandGuide/FileCheck.html#filecheck-variables)
for
later
matching
,
and
[
checking
non
-
sequential
matches
](
https
:
//llvm.org/docs/CommandGuide/FileCheck.html#the-check-dag-directive).
####
Capturing
Variables
For
example
,
say
GCC
stores
a
variable
in
a
register
but
Clang
stores
it
in
memory
.
To
write
a
test
that
tolerates
both
cases
we
"capture"
the
destination
of
the
store
,
and
then
use
the
captured
expression
to
write
the
remainder
of
the
test
.
```
c
++
// CHECK-LABEL: test_div_no_op_into_shr:
extern
"C"
void
test_div_no_op_into_shr
(
int
value
)
{
int
divisor
=
2
;
benchmark
::
DoNotOptimize
(
divisor
);
// hide the value from the optimizer
return
value
/
divisor
;
// CHECK: movl $2, [[DEST:.*]]
// CHECK: idivl [[DEST]]
// CHECK: ret
}
```
####
Using
Regular
Expressions
to
Match
Differing
Output
Often
tests
require
testing
assembly
lines
which
may
subtly
differ
between
compilers
or
compiler
versions
.
A
common
example
of
this
is
matching
stack
frame
addresses
.
In
this
case
regular
expressions
can
be
used
to
match
the
differing
bits
of
output
.
For
example
:
```
c
++
int
ExternInt
;
struct
Point
{
int
x
,
y
,
z
;
};
// CHECK-LABEL: test_store_point:
extern
"C"
void
test_store_point
()
{
Point
p
{
ExternInt
,
ExternInt
,
ExternInt
};
benchmark
::
DoNotOptimize
(
p
);
// CHECK: movl ExternInt(%rip), %eax
// CHECK: movl %eax, -{{[0-9]+}}(%rsp)
// CHECK: movl %eax, -{{[0-9]+}}(%rsp)
// CHECK: movl %eax, -{{[0-9]+}}(%rsp)
// CHECK: ret
}
```
##
Current
Requirements
and
Limitations
The
tests
require
Filecheck
to
be
installed
along
the
`PATH`
of
the
build
machine
.
Otherwise
the
tests
will
be
disabled
.
Additionally
,
as
mentioned
in
the
previous
section
,
codegen
tests
are
inherently
non
-
portable
.
Currently
the
tests
are
limited
to
:
*
x86_64
targets
.
*
Compiled
with
GCC
or
Clang
Further
work
could
be
done
,
at
least
on
a
limited
basis
,
to
extend
the
tests
to
other
architectures
and
compilers
(
using
`CHECK`
prefixes
).
Furthermore
,
the
tests
fail
for
builds
which
specify
additional
flags
that
modify
code
generation
,
including
`--coverage` or `-fsanitize=`
.
Event Timeline
Log In to Comment