Commit Graph

361 Commits

Author SHA1 Message Date
Sergey Fedorov
e60c24cecf
tcg: Clean up direct block chaining data fields
Briefly describe in a comment how direct block chaining is done. It
should help in understanding of the following data fields.

Rename some fields in TranslationBlock and TCGContext structures to
better reflect their purpose (dropping excessive 'tb_' prefix in
TranslationBlock but keeping it in TCGContext):
tb_next_offset => jmp_reset_offset
tb_jmp_offset => jmp_insn_offset
tb_next => jmp_target_addr
jmp_next => jmp_list_next
jmp_first => jmp_list_first

Avoid using a magic constant as an invalid offset which is used to
indicate that there's no n-th jump generated.

Backports commit f309101c26b59641fc1aa8fb2a98a5441cdaea03 from qemu
2018-02-23 21:28:19 -05:00
Sergey Fedorov
c5b234ed1f
tcg: Note requirement on atomic direct jump patching
Backports commit 10b4f4855537dd421e193a7d0416513116370558 from qemu
2018-02-23 21:28:18 -05:00
Sergey Fedorov
52e2972300
tcg/arm: Make direct jump patching thread-safe
Ensure direct jump patching in ARM is atomic by using
atomic_read()/atomic_set() for code patching.

Backports commit 7d14e0e2d661479985197203589c38840e1066df from qemu
2018-02-23 21:28:18 -05:00
Sergey Fedorov
57359fbe6c
tcg/s390: Make direct jump patching thread-safe
Ensure direct jump patching in s390 is atomic by:
* naturally aligning a location of direct jump address;
* using atomic_read()/atomic_set() for code patching.

Backports commit ed3d51ecd7fe248d3959e469d53890ac9ffe0cd2 from qemu
2018-02-23 21:28:18 -05:00
Sergey Fedorov
5eb2d6618f
tcg/i386: Make direct jump patching thread-safe
Ensure direct jump patching in i386 is atomic by:
* naturally aligning a location of direct jump address;
* using atomic_read()/atomic_set() for code patching.

Backports commit 0d07abf05e98903c7faf204a9a90f7d45b7554dc from qemu
2018-02-23 21:28:17 -05:00
Lioncash
fffa27d269
osdep: MSVC-compatible alignment macros 2018-02-23 21:28:17 -05:00
Sergey Fedorov
3456f0879e
include/qemu/osdep.h: Add macros for pointer alignment
These macros provide a convenient way to n-byte align pointers up and
down and check if a pointer is n-byte aligned.

Backports commit 6b587d3cda48e7ba26de8d30bf0d8a7063970715 from qemu
2018-02-23 21:28:17 -05:00
Sergey Fedorov
47eac70cb9
include/qemu/osdep.h: Add a macro to check for alignment
Backports commit 18a60a76147569ca9e11b0607e50ce4012fe1aaa from qemu
2018-02-23 21:28:17 -05:00
Emilio G. Cota
170f6e0b3b
tb: consistently use uint32_t for tb->flags
We are inconsistent with the type of tb->flags: usage varies loosely
between int and uint64_t. Settle to uint32_t everywhere, which is
superior to both: at least one target (aarch64) uses the most significant
bit in the u32, and uint64_t is wasteful.

Compile-tested for all targets.

Backports commit 89fee74a0f066dfd73830a7b5fa137e87888c870 from qemu
2018-02-23 21:28:11 -05:00
Edgar E. Iglesias
bfc74c4da2
gen-icount: Use tcg_set_insn_param
Use tcg_set_insn_param() instead of directly accessing internal
tcg data structures to update an insn param.

Backports commit 25caa94c4a26daaab1e65c6d887e2972aeb5749e from qemu
2018-02-23 20:01:17 -05:00
Eric Blake
2f42c2c195
qapi: Change visit_type_FOO() to no longer return partial objects
Returning a partial object on error is an invitation for a careless
caller to leak memory. We already fixed things in an earlier
patch to guarantee NULL if visit_start fails ("qapi: Guarantee
NULL obj on input visitor callback error"), but that does not
help the case where visit_start succeeds but some other failure
happens before visit_end, such that we leak a partially constructed
object outside visit_type_FOO(). As no one outside the testsuite
was actually relying on these semantics, it is cleaner to just
document and guarantee that ALL pointer-based visit_type_FOO()
functions always leave a safe value in *obj during an input visitor
(either the new object on success, or NULL if an error is
encountered), so callers can now unconditionally use
qapi_free_FOO() to clean up regardless of whether an error occurred.

The decision is done by adding visit_is_input(), then updating the
generated code to check if additional cleanup is needed based on
the type of visitor in use.

Note that we still leave *obj unchanged after a scalar-based
visit_type_FOO(); I did not feel like auditing all uses of
visit_type_Enum() to see if the callers would tolerate a specific
sentinel value (not to mention having to decide whether it would
be better to use 0 or ENUM__MAX as that sentinel).

Backports commit 68ab47e4b4ecc1c4649362b8cc1e49794d1a6537 from qemu
2018-02-23 19:53:17 -05:00
Eric Blake
0d52542da2
qapi: Simplify semantics of visit_next_list()
The semantics of the list visit are somewhat baroque, with the
following pseudocode when FooList is used:

start()
for (prev = head; cur = next(prev); prev = &cur) {
visit(&cur->value)
}

Note that these semantics (advance before visit) requires that
the first call to next() return the list head, while all other
calls return the next element of the list; that is, every visitor
implementation is required to track extra state to decide whether
to return the input as-is, or to advance. It also requires an
argument of 'GenericList **' to next(), solely because the first
iteration might need to modify the caller's GenericList head, so
that all other calls have to do a layer of dereferencing.

Thankfully, we only have two uses of list visits in the entire
code base: one in spapr_drc (which completely avoids
visit_next_list(), feeding in integers from a different source
than uint8List), and one in qapi-visit.py. That is, all other
list visitors are generated in qapi-visit.c, and share the same
paradigm based on a qapi FooList type, so we can refactor how
lists are laid out with minimal churn among clients.

We can greatly simplify things by hoisting the special case
into the start() routine, and flipping the order in the loop
to visit before advance:

start(head)
for (tail = *head; tail; tail = next(tail)) {
visit(&tail->value)
}

With the simpler semantics, visitors have less state to track,
the argument to next() is reduced to 'GenericList *', and it
also becomes obvious whether an input visitor is allocating a
FooList during visit_start_list() (rather than the old way of
not knowing if an allocation happened until the first
visit_next_list()). As a minor drawback, we now allocate in
two functions instead of one, and have to pass the size to
both functions (unless we were to tweak the input visitors to
cache the size to start_list for reuse during next_list, but
that defeats the goal of less visitor state).

The signature of visit_start_list() is chosen to match
visit_start_struct(), with the new parameters after 'name'.

The spapr_drc case is a virtual visit, done by passing NULL for
list, similarly to how NULL is passed to visit_start_struct()
when a qapi type is not used in those visits. It was easy to
provide these semantics for qmp-output and dealloc visitors,
and a bit harder for qmp-input (several prerequisite patches
refactored things to make this patch straightforward). But it
turned out that the string and opts visitors munge enough other
state during visit_next_list() to make it easier to just
document and require a GenericList visit for now; an assertion
will remind us to adjust things if we need the semantics in the
future.

Several pre-requisite cleanup patches made the reshuffling of
the various visitors easier; particularly the qmp input visitor.

Backports commit d9f62dde1303286b24ac8ce88be27e2b9b9c5f46 from qemu
2018-02-23 19:50:26 -05:00
Eric Blake
6084be1882
qapi: Split visit_end_struct() into pieces
As mentioned in previous patches, we want to call visit_end_struct()
functions unconditionally, so that visitors can release resources
tied up since the matching visit_start_struct() without also having
to worry about error priority if more than one error occurs.

Even though error_propagate() can be safely used to ignore a second
error during cleanup caused by a first error, it is simpler if the
cleanup cannot set an error. So, split out the error checking
portion (basically, input visitors checking for unvisited keys) into
a new function visit_check_struct(), which can be safely skipped if
any earlier errors are encountered, and leave the cleanup portion
(which never fails, but must be called unconditionally if
visit_start_struct() succeeded) in visit_end_struct().

Generated code in qapi-visit.c has diffs resembling:

|@@ -59,10 +59,12 @@ void visit_type_ACPIOSTInfo(Visitor *v,
| goto out_obj;
| }
| visit_type_ACPIOSTInfo_members(v, obj, &err);
|- error_propagate(errp, err);
|- err = NULL;
|+ if (err) {
|+ goto out_obj;
|+ }
|+ visit_check_struct(v, &err);
| out_obj:
|- visit_end_struct(v, &err);
|+ visit_end_struct(v);
| out:

and in qapi-event.c:

@@ -47,7 +47,10 @@ void qapi_event_send_acpi_device_ost(ACP
| goto out;
| }
| visit_type_q_obj_ACPI_DEVICE_OST_arg_members(v, &param, &err);
|- visit_end_struct(v, err ? NULL : &err);
|+ if (!err) {
|+ visit_check_struct(v, &err);
|+ }
|+ visit_end_struct(v);
| if (err) {
| goto out;

Backports commit 15c2f669e3fb2bc97f7b42d1871f595c0ac24af8 from qemu
2018-02-23 19:13:47 -05:00
Eric Blake
ef6b7b50f6
qapi: Add visit_type_null() visitor
Right now, qmp-output-visitor happens to produce a QNull result
if nothing is actually visited between the creation of the visitor
and the request for the resulting QObject. A stronger protocol
would require that a QMP output visit MUST visit something. But
to still be able to produce a JSON 'null' output, we need a new
visitor function that states our intentions. Yes, we could say
that such a visit must go through visit_type_any(), but that
feels clunky.

So this patch introduces the new visit_type_null() interface and
its no-op interface in the dealloc visitor, and stubs in the
qmp visitors (the next patch will finish the implementation).
For the visitors that will not implement the callback, document
the situation. The code in qapi-visit-core unconditionally
dereferences the callback pointer, so that a segfault will inform
a developer if they need to implement the callback for their
choice of visitor.

Note that JSON has a primitive null type, with the single value
null; likewise with the QNull type for QObject; but for QAPI,
we just have the 'null' value without a null type. We may
eventually want to add more support in QAPI for null (most likely,
we'd use it via an alternate type that permits 'null' or an
object); but we'll create that usage when we need it.

Backports commit 3bc97fd5924561d92f32758c67eaffd2e4e25038 from qemu
2018-02-23 15:48:57 -05:00
Eric Blake
fafb3e354b
qapi: Document visitor interfaces, add assertions
The visitor interface for mapping between QObject/QemuOpts/string
and QAPI is scandalously under-documented, making changes to visitor
core, individual visitors, and users of visitors difficult to
coordinate. Among other questions: when is it safe to pass NULL,
vs. when a string must be provided; which visitors implement which
callbacks; the difference between concrete and virtual visits.

Correct this by retrofitting proper contracts, and document where some
of the interface warts remain (for example, we may want to modify
visit_end_* to require the same 'obj' as the visit_start counterpart,
so the dealloc visitor can be simplified). Later patches in this
series will tackle some, but not all, of these warts.

Add assertions to (partially) enforce the contract. Some of these
were only made possible by recent cleanup commits.

Backports commit adfb264c9ed04bfc694921b72173be8e29e90024 from qemu
2018-02-23 15:45:31 -05:00
Eric Blake
9e999acc83
qapi: Change visit_start_implicit_struct to visit_start_alternate
After recent changes, the only remaining use of
visit_start_implicit_struct() is for allocating the space needed
when visiting an alternate. Since the term 'implicit struct' is
hard to explain, rename the function to its current usage. While
at it, we can merge the functionality of visit_get_next_type()
into the same function, making it more like visit_start_struct().

Generated code is now slightly smaller:

| {
| Error *err = NULL;
|
|- visit_start_implicit_struct(v, (void**) obj, sizeof(BlockdevRef), &err);
|+ visit_start_alternate(v, name, (GenericAlternate **)obj, sizeof(**obj),
|+ true, &err);
| if (err) {
| goto out;
| }
|- visit_get_next_type(v, name, &(*obj)->type, true, &err);
|- if (err) {
|- goto out_obj;
|- }
| switch ((*obj)->type) {
| case QTYPE_QDICT:
| visit_start_struct(v, name, NULL, 0, &err);
...
| }
|-out_obj:
|- visit_end_implicit_struct(v);
|+ visit_end_alternate(v);
| out:
| error_propagate(errp, err);
| }

Backports commit dbf11922622685934bfb41e7cf2be9bd4a0405c0 from qemu
2018-02-23 15:33:25 -05:00
Eric Blake
559304aed9
qapi: Consolidate QMP input visitor creation
Rather than having two separate ways to create a QMP input
visitor, where the safer approach has the more verbose name,
it is better to consolidate things into a single function
where the caller must explicitly choose whether to be strict
or to ignore excess input. This patch is the strictly
mechanical conversion; the next patch will then audit which
uses can be made stricter.

Backports commit fc471c18d5d2ec713d5a019f9530398675494bc8 from qemu
2018-02-23 15:09:57 -05:00
Eric Blake
3cf7b6dd3b
qapi: Adjust layout of FooList types
By sticking the next pointer first, we don't need a union with
64-bit padding for smaller types.  On 32-bit platforms, this
can reduce the size of uint8List from 16 bytes (or 12, depending
on whether 64-bit ints can tolerate 4-byte alignment) down to 8.
It has no effect on 64-bit platforms (where alignment still
dictates a 16-byte struct); but fewer anonymous unions is still
a win in my book.

It requires visit_next_list() to gain a size parameter, to know
what size element to allocate; comparable to the size parameter
of visit_start_struct().

I debated about going one step further, to allow for fewer casts,
by doing:
    typedef GenericList GenericList;
    struct GenericList {
        GenericList *next;
    };
    struct FooList {
        GenericList base;
        Foo *value;
    };
so that you convert to 'GenericList *' by '&foolist->base', and
back by 'container_of(generic, GenericList, base)' (as opposed to
the existing '(GenericList *)foolist' and '(FooList *)generic').
But doing that would require hoisting the declaration of
GenericList prior to inclusion of qapi-types.h, rather than its
current spot in visitor.h; it also makes iteration a bit more
verbose through 'foolist->base.next' instead of 'foolist->next'.

Note that for lists of objects, the 'value' payload is still
hidden behind a boxed pointer.  Someday, it would be nice to do:

struct FooList {
    FooList *next;
    Foo value;
};

for one less level of malloc for each list element.  This patch
is a step in that direction (now that 'next' is no longer at a
fixed non-zero offset within the struct, we can store more than
just a pointer's-worth of data as the value payload), but the
actual conversion would be a task for another series, as it will
touch a lot of code.

Backports commit e65d89bf1a4484e0db0f3dc820a8b209f2fb1e8b from qemu
2018-02-23 14:49:06 -05:00
Eric Blake
eef0932471
qapi-visit: Add visitor.type classification
We have three classes of QAPI visitors: input, output, and dealloc.
Currently, all implementations of these visitors have one thing in
common based on their visitor type: the implementation used for the
visit_type_enum() callback. But since we plan to add more such
common behavior, in relation to documenting and further refining
the semantics, it makes more sense to have the visitor
implementations advertise which class they belong to, so the common
qapi-visit-core code can use that information in multiple places.

A later patch will better document the types of visitors directly
in visitor.h.

For this patch, knowing the class of a visitor implementation lets
us make input_type_enum() and output_type_enum() become static
functions, by replacing the callback function Visitor.type_enum()
with the simpler enum member Visitor.type. Share a common
assertion in qapi-visit-core as part of the refactoring.

Move comments in opts-visitor.c to match the refactored layout.

Backports commit 983f52d4b3f86fb9dc9f8b142132feb5a8723016 from qemu
2018-02-23 14:25:41 -05:00
Fam Zheng
5c739f14f5
util: Fix MIN_NON_ZERO
MIN_NON_ZERO(1, 0) is evaluated to 0. Rewrite the macro to fix it.

Backports commit b6ece2c6f37926a994bc564a9e55ef3be6016d8f from qemu
2018-02-23 14:09:44 -05:00
Lioncash
87130fc884
exec-all: Remove externs
These are unused
2018-02-23 12:43:03 -05:00
Peter Crosthwaite
576f1752a6
include/exec: Move cputlb exec.c defs out
Move the architecture agnostic function prototypes for exec.c out of
cputlb.h to exec-all.h. This allows hiding of the arch specific
cputlb.h from exec.c which should be getting close to having no
architecture specifics. Prepares support for multi-arch, which will have
a minimal cpu.h that services exec.c but not cputlb.h.

Backports commit dfccc7602374c9fd3b083208b552d62daa244811 from qemu
2018-02-23 10:52:25 -05:00
Peter Crosthwaite
97c9423ee8
cputlb: move CPU_LOOP() for tlb_reset() to exec.c
To prepare for multi-arch, cputlb.c should only have awareness of one
single architecture. This means it should not have access to the full
CPU lists which may be heterogeneous. Instead, push the CPU_LOOP() up
to the one and only caller in exec.c.

Backports commit 9a13565d52bfd321934fb44ee004bbaf5f5913a8 from qemu
2018-02-23 10:46:31 -05:00
Paolo Bonzini
9479199c6b
memory: fix usage of find_next_bit and find_next_zero_bit
The last two arguments to these functions are the last and first bit to
check relative to the base. The code was using incorrectly the first
bit and the number of bits. Fix this in cpu_physical_memory_get_dirty
and cpu_physical_memory_all_dirty. This requires a few changes in the
iteration; change the code in cpu_physical_memory_set_dirty_range to
match.

Backports commit 88c73d16ad1b6c22a2ab082064d0d521f756296a from qemu
2018-02-22 19:51:43 -05:00
Alex Bennée
171d267209
include/qemu/atomic.h: default to __atomic functions
The __atomic primitives have been available since GCC 4.7 and provide
a richer interface for describing memory ordering requirements. As a
bonus by using the primitives instead of hand-rolled functions we can
use tools such as the ThreadSanitizer which need the use of well
defined APIs for its analysis.

If we have __ATOMIC defines we exclusively use the __atomic primitives
for all our atomic access. Otherwise we fall back to the mixture of
__sync and hand-rolled barrier cases.

Backports commit a0aa44b488b3601415d55041e4619aef5f3a4ba8 from qemu
2018-02-22 16:12:59 -05:00
Paolo Bonzini
4e7259a49b
atomics: add explicit compiler fence in __atomic memory barriers
__atomic_thread_fence does not include a compiler barrier; in the
C++11 memory model, fences take effect in combination with other
atomic operations.  GCC implements this by making __atomic_load and
__atomic_store access memory as if the pointer was volatile, and
leaves no trace whatsoever of acquire and release fences in the
compiler's intermediate representation.

In QEMU, we want memory barriers to act on all memory, but at the same
time we would like to use __atomic_thread_fence for portability reasons.
Add compiler barriers manually around the __atomic_thread_fence.

Backports commit 3bbf572345c65813f86a8fc434ea1b23beb08e16 from qemu
2018-02-22 15:56:37 -05:00
Paolo Bonzini
02e3eeff40
atomic: fix position of volatile qualifier
What needs to be volatile is not the pointer, but the pointed-to
value!

Backports commit 2cbcfb281afa041a41f6e4c4da0f5c9314084604 from qemu
2018-02-22 15:52:48 -05:00
Stefan Hajnoczi
e79e0881cd
memory: RCU ram_list.dirty_memory[] for safe RAM hotplug
Although accesses to ram_list.dirty_memory[] use atomics so multiple
threads can safely dirty the bitmap, the data structure is not fully
thread-safe yet.

This patch handles the RAM hotplug case where ram_list.dirty_memory[] is
grown.  ram_list.dirty_memory[] is change from a regular bitmap to an
RCU array of pointers to fixed-size bitmap blocks.  Threads can continue
accessing bitmap blocks while the array is being extended.  See the
comments in the code for an in-depth explanation of struct
DirtyMemoryBlocks.

I have tested that live migration with virtio-blk dataplane works.

Backports commit 5b82b703b69acc67b78b98a5efc897a3912719eb from qemu
2018-02-22 15:38:03 -05:00
Denis V. Lunev
eb29ff04ca
log: move qemu_log_close/qemu_log_flush from header to log.c
There is no particular reason to keep these functions in the header.
Suggested by Paolo.

Backports commit 99affd1d5bd4e396ecda50e53dfbc5147fa1313d from qemu
2018-02-22 11:13:17 -05:00
Alex Bennée
3da7d9d9ae
qemu-log: dfilter-ise exec, out_asm, op and opt_op
qemu-log: dfilter-ise exec, out_asm, op and opt_op

This ensures the code generation debug code will honour -dfilter if set.
For the "exec" tracing I've added a new inline macro for efficiency's
sake.

Backports commit d977e1c2dbc9e63454b2000f91954d02543bf43b from qemu
2018-02-22 10:06:19 -05:00
Alex Bennée
2d401b6f23
qemu-log: new option -dfilter to limit output
When debugging big programs or system emulation sometimes you want both
the verbosity of cpu,exec et all but don't want to generate lots of logs
for unneeded stuff. This patch adds a new option -dfilter which allows
you to specify interesting address ranges in the form:

-dfilter 0x8000..0x8fff,0xffffffc000080000+0x200,...

Then logging code can use the new qemu_log_in_addr_range() function to
decide if it will output logging information for the given range.

Backports commit 3514552e04388d8e7686bcf89efd022e892acb5b from qemu
2018-02-22 10:02:26 -05:00
Peter Maydell
3f5e36e15f
qemu-log: Improve the exec TB execution logging
Improve the TB execution logging so that it is easier to identify
what is happening from trace logs:
* move the "Trace" logging of executed TBs into cpu_tb_exec()
so that it is emitted if and only if we actually execute a TB,
and for consistency for the CPU state logging
* log when we link two TBs together via tb_add_jump()
* log when cpu_tb_exec() returns early from a chain of TBs

The new style logging looks like this:

Trace 0x7fb7cc822ca0 [ffffffc0000dce00]
Linking TBs 0x7fb7cc822ca0 [ffffffc0000dce00] index 0 -> 0x7fb7cc823110 [ffffffc0000dce10]
Trace 0x7fb7cc823110 [ffffffc0000dce10]
Trace 0x7fb7cc823420 [ffffffc000302688]
Trace 0x7fb7cc8234a0 [ffffffc000302698]
Trace 0x7fb7cc823520 [ffffffc0003026a4]
Trace 0x7fb7cc823560 [ffffffc0000dce44]
Linking TBs 0x7fb7cc823560 [ffffffc0000dce44] index 1 -> 0x7fb7cc8235d0 [ffffffc0000dce70]
Trace 0x7fb7cc8235d0 [ffffffc0000dce70]
Stopped execution of TB chain before 0x7fb7cc8235d0 [ffffffc0000dce70]
Trace 0x7fb7cc8235d0 [ffffffc0000dce70]
Trace 0x7fb7cc822fd0 [ffffffc0000dd52c]

Backports commit 1a830635229e14c403600167823ea6b3b79d3097 from qemu
2018-02-22 09:40:11 -05:00
Peter Maydell
66e1bacd64
qemu-log: Avoid function call for disabled qemu_log_mask logging
Make qemu_log_mask() a macro which only calls the function to
do the actual work if the logging is enabled. This avoids making
a function call in possible fast paths where logging is disabled.

Backports commit 7ee606230e6b7645d92365d9b39179368e83ac54 from qemu
2018-02-22 09:32:48 -05:00
Veronia Bahaa
bafc81b1d3
util: move declarations out of qemu-common.h
Move declarations out of qemu-common.h for functions declared in
utils/ files: e.g. include/qemu/path.h for utils/path.c.
Move inline functions out of qemu-common.h and into new files (e.g.
include/qemu/bcd.h)

Backports commit f348b6d1a53e5271cf1c9f9acc4646b4b98c1771 from qemu
2018-02-22 09:25:48 -05:00
Marc-André Lureau
fff79ed49b
utils: rename strtosz to use qemu prefix
Not only it makes sense, but it gets rid of checkpatch warning:
WARNING: consider using qemu_strtosz in preference to strtosz

Also remove get rid of tabs to please checkpatch.

Backports commit 4677bb40f809394bef5fa07329dea855c0371697 from qemu
2018-02-22 00:17:52 -05:00
Rutuja Shah
d9fdc180d7
Replaced get_tick_per_sec() by NANOSECONDS_PER_SECOND
This patch replaces get_ticks_per_sec() calls with the macro
NANOSECONDS_PER_SECOND. Also, as there are no callers, get_ticks_per_sec()
is then removed. This replacement improves the readability and
understandability of code.

For example,

timer_mod(fdctrl->result_timer,
qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) + (get_ticks_per_sec() / 50));

NANOSECONDS_PER_SECOND makes it obvious that qemu_clock_get_ns
matches the unit of the expression on the right side of the plus.

Backports commit 73bcb24d932912f8e75e1d88da0fc0ac6d4bce78 from qemu
2018-02-21 23:21:36 -05:00
Markus Armbruster
6730bd3131
Move QEMU_ALIGN_*() from qemu-common.h to qemu/osdep.h
qemu-common.h should only be included by .c files. Its file comment
explains why: "No header file should depend on qemu-common.h, as this
would easily lead to circular header dependencies."

One of the reasons for headers to include it is QEMU_ALIGN_UP() and
QEMU_ALIGN_DOWN(). Move them next to ROUND_UP() in qemu/osdep.h, to
facilitate removing these ill-advised includes later on.

Backports commit e07e540aaa08718c9ff8213067a3dcef31b3e313 from qemu
2018-02-21 23:12:24 -05:00
Markus Armbruster
6b1ebd16e6
Move HOST_LONG_BITS from qemu-common.h to qemu/osdep.h
qemu-common.h should only be included by .c files. Its file comment
explains why: "No header file should depend on qemu-common.h, as this
would easily lead to circular header dependencies."

One of the reasons for headers to include it is HOST_LONG_BITS. Move
that to its more natural home qemu/osdep.h, to facilitate removing
these ill-advised includes later on.

This also lets us use HOST_LONG_BITS in bswap.h instead of duplicating
its definition there to avoid cyclic inclusion.

Backports commit a8139632161d7546218b696cada0a4f64cc78fb7 from qemu
2018-02-21 23:10:43 -05:00
Markus Armbruster
06668850e3
include/qemu/osdep.h: Don't include qapi/error.h
Commit 57cb38b included qapi/error.h into qemu/osdep.h to get the
Error typedef. Since then, we've moved to include qemu/osdep.h
everywhere. Its file comment explains: "To avoid getting into
possible circular include dependencies, this file should not include
any other QEMU headers, with the exceptions of config-host.h,
compiler.h, os-posix.h and os-win32.h, all of which are doing a
similar job to this file and are under similar constraints."
qapi/error.h doesn't do a similar job, and it doesn't adhere to
similar constraints: it includes qapi-types.h. That's in excess of
100KiB of crap most .c files don't actually need.

Add the typedef to qemu/typedefs.h, and include that instead of
qapi/error.h. Include qapi/error.h in .c files that need it and don't
get it now. Include qapi-types.h in qom/object.h for uint16List.

Update scripts/clean-includes accordingly. Update it further to match
reality: replace config.h by config-target.h, add sysemu/os-posix.h,
sysemu/os-win32.h. Update the list of includes in the qemu/osdep.h
comment quoted above similarly.

This reduces the number of objects depending on qapi/error.h from "all
of them" to less than a third. Unfortunately, the number depending on
qapi-types.h shrinks only a little. More work is needed for that one.

Backports commit da34e65cb4025728566d6504a99916f6e7e1dd6a from qemu
2018-02-21 23:08:18 -05:00
Stefan Weil
baa477d324
Remove unneeded include statements for setjmp.h
As soon as setjmp.h is included from qemu/osdep.h, those old include
statements are no longer needed.

Add also setjmp.h to the list in scripts/clean-includes.

Backports commit 8ff98f1ed2f50cd05c3c5027c7efdf69859ec664 from qemu
2018-02-21 22:57:32 -05:00
Stefan Weil
904b3c467e
Include setjmp.h in qemu/osdep.h (bug fix for w64)
setjmp must be declared before sysemu/os-win32.h
because it is redefined there for 64 bit Windows.

Backports commit e89fdafb58038038e3ccb860c5e1068ba063bac8 from qemu
2018-02-21 22:56:46 -05:00
Max Reitz
1cfdf802a9
qapi: Drop QERR_UNKNOWN_BLOCK_FORMAT_FEATURE
Just specifying a custom string is simpler in basically all places that
used it, and in addition, specifying the BB or node name is something we
generally do not do in other error messages when opening a BDS, so we
should not do it here.

This changes the output for iotest 036 (to the better, in my opinion),
so the reference output needs to be changed accordingly.

Backports commit a55448b3681a880b77eaefe8b2c42912000cb481 from qemu
2018-02-21 21:55:15 -05:00
Daniel P. Berrange
eddfb13c2c
qom: Change object property iterator API contract
Currently the ObjectProperty iterator API works as follows:

ObjectPropertyIterator *iter;

iter = object_property_iter_init(obj);
while ((prop = object_property_iter_next(iter))) {
...
}
object_property_iter_free(iter);

This has the benefit that the ObjectPropertyIterator struct
can be opaque, but has the downside that callers need to
explicitly call a free function. It is also not in keeping
with iterator style used elsewhere in QEMU/GLib2.

This patch changes the API to use stack allocation instead:

ObjectPropertyIterator iter;

object_property_iter_init(&iter, obj);
while ((prop = object_property_iter_next(&iter))) {
...
}

Backports commit 7746abd8e9ee9db20c0b0fdb19504f163ba3cbea from qemu
2018-02-21 21:03:58 -05:00
Daniel P. Berrange
b97ab59f08
qom: Allow properties to be registered against classes
When there are many instances of a given class, registering
properties against the instance is wasteful of resources. The
majority of objects have a statically defined list of possible
properties, so most of the properties are easily registerable
against the class. Only those properties which are conditionally
registered at runtime need be recorded against the klass.

Registering properties against classes also makes it possible
to provide static introspection of QOM - currently introspection
is only possible after creating an instance of a class, which
severely limits its usefulness.

This impl only supports simple scalar properties. It does not
attempt to allow child object / link object properties against
the class. There are ways to support those too, but it would
make this patch more complicated, so it is left as an exercise
for the future.

There is no equivalent to object_property_del() provided, since
classes must be immutable once they are defined.

Backports commit 16bf7f522a2ff68993f80631ed86254c71eaf5d4 from qemu
2018-02-21 21:00:56 -05:00
Pavel Fedin
825bc2fb04
qom: Replace object property list with GHashTable
ARM GICv3 systems with large number of CPUs create lots of IRQ pins. Since
every pin is represented as a property, number of these properties becomes
very large. Every property add first makes sure there's no duplicates.
Traversing the list becomes very slow, therefore QEMU initialization takes
significant time (several seconds for e. g. 16 CPUs).

This patch replaces list with GHashTable, making lookup very fast. The only
drawback is that object_child_foreach() and object_child_foreach_recursive()
cannot add or remove properties during traversal, since GHashTableIter does
not have modify-safe version. However, the code seems not to modify objects
via these functions.

Backports commit b604a854e843505007c59d68112c654556102a20 from qemu
2018-02-21 13:35:10 -05:00
Lioncash
4b79ff71b4
glib_compat: backport hashtable iterator interfaces 2018-02-21 13:18:44 -05:00
Pavel Fedin
0201c71145
Merge memory_region_init_reservation() into memory_region_init_io()
Just specifying ops = NULL in some cases can be more convenient than having
two functions.

Backports commit 6d6d2abf2c2e52c0f404d0a31a963e945b0cc7ad from qemu
2018-02-21 11:23:00 -05:00
Fam Zheng
fa7d3e6cdb
memory: Drop MemoryRegion.ram_addr
All references to mr->ram_addr are replaced by
memory_region_get_ram_addr(mr) (except for a few assertions that are
replaced with mr->ram_block).

Backports commit 8e41fb63c5bf29ecabe0cee1239bf6230f19978a from qemu
2018-02-21 08:53:08 -05:00
Fam Zheng
2c1a72635d
memory: Implement memory_region_get_ram_addr with mr->ram_block
Backports commit 7ebb2745acbb8d910eab07dc5f0aa01a4457703c from qemu
2018-02-21 08:53:08 -05:00
Gonglei
aa80edbef0
exec: Return RAMBlock pointer from allocating functions
Previously we return RAMBlock.offset; now return the pointer to the
whole structure.

ram_block_add returns void now, error is completely passed with errp.

Backports commit 528f46af6ecd1e300db18684969104d4067b867b from qemu
2018-02-21 08:52:57 -05:00
Lluís Vilanova
d111e2df2d
typedefs: Add CPUState
Backports commit b23197f9cf2f221a6cc6272d36852f4f70cf9c1b from qemu
2018-02-21 01:55:22 -05:00
Gonglei
26951bf754
memory: Remove unreachable return statement
Backports commit d61524486c6e503e502241a2ea834f930f98a6a1 from qemu
2018-02-20 20:54:24 -05:00
Gonglei
d25285bc78
memory: optimize qemu_get_ram_ptr and qemu_ram_ptr_length
these two functions consume too much cpu overhead to
find the RAMBlock by ram address.

After this patch, we can pass the RAMBlock pointer
to them so that they don't need to find the RAMBlock
anymore most of the time. We can get better performance
in address translation processing.

Backports commit 3655cb9c7375a595a8051ec677c515b24d5c1fe6 from qemu
2018-02-20 20:53:31 -05:00
Gonglei
39e4d63e68
exec: store RAMBlock pointer into memory region
Each RAM memory region has a unique corresponding RAMBlock.
In the current realization, the memory region only stored
the ram_addr which means the offset of RAM address space,
We need to qurey the global ram.list to find the ram block
by ram_addr if we want to get the ram block, which is very
expensive.

Now, we store the RAMBlock pointer into memory region
structure. So, if we know the mr, we can easily get the
RAMBlock.

Backports commit 58eaa2174e99d9a05172d03fd2799ab8fd9e6f60 from qemu
2018-02-20 20:43:32 -05:00
Peter Maydell
547fabd58e
osdep.h: Include config-target.h if NEED_CPU_H is defined
NEED_CPU_H is the define we use to distinguish per-target object
compilation from common object compilation. For the former, we must
also include config-target.h so that the .c files see the necessary
CONFIG_ constants.

Backports commit b1e34d1c3a9059e87719634bfc4db53174d63e14 from qemu
2018-02-20 19:11:07 -05:00
Peter Maydell
c41bb9a772
osdep.h: Define macros for the benefit of C++ before C++11
For C++ before C++11, <stdint.h> requires definition of the macros
__STDC_CONSTANT_MACROS, __STDC_LIMIT_MACROS and __STDC_FORMAT_MACROS
in order to enable definition of various macros by the header file.
Define these in osdep.h, so that we get the right header file
definitions whether osdep.h is being used by plain C, C++11 or
older C++.

In particular libvixl's header files depend on this and won't
compile if osdep.h is included before them otherwise.

Backports commit 79f56d82f805b170fa2be8c04b682117be56483f from qemu
2018-02-20 19:09:58 -05:00
Lioncash
c17fa2cad3
osdep.h: Remove int_fast*_t Solaris compatibility code
We now do not use the int_fast*_t types anywhere in QEMU, so we can
remove the compatibility definitions we were providing for the
benefit of ancient Solaris versions.

Backports commit 50fe4df8ee6aba63ae51457bad40ba26e3c9746f from qemu
2018-02-20 18:58:53 -05:00
Peter Maydell
68cbe1b2ce
fpu: Remove use of int_fast16_t in conversions to int16
Make the functions which convert floating point to 16 bit integer
return int16_t rather than int_fast16_t, and correspondingly use
int_fast16_t in their internal implementations where appropriate.

(These functions are used only by the ARM target.)

Backports commit 0bb721d7217ed4a1abb44f521c5c7ec185062d58 from qemu
2018-02-20 16:54:04 -05:00
Eric Blake
e096e62127
qapi: Don't box branches of flat unions
There's no reason to do two malloc's for a flat union; let's just
inline the branch struct directly into the C union branch of the
flat union.

Surprisingly, fewer clients were actually using explicit references
to the branch types in comparison to the number of flat unions
thus modified.

This lets us reduce the hack in qapi-types:gen_variants() added in
the previous patch; we no longer need to distinguish between
alternates and flat unions.

The change to unboxed structs means that u.data (added in commit
cee2dedb) is now coincident with random fields of each branch of
the flat union, whereas beforehand it was only coincident with
pointers (since all branches of a flat union have to be objects).
Note that this was already the case for simple unions - but there
we got lucky. Remember, visit_start_union() blindly returns true
for all visitors except for the dealloc visitor, where it returns
the value !!obj->u.data, and that this result then controls
whether to proceed with the visit to the variant. Pre-patch,
this meant that flat unions were testing whether the boxed pointer
was still NULL, and thereby skipping visit_end_implicit_struct()
and avoiding a NULL dereference if the pointer had not been
allocated. The same was true for simple unions where the current
branch had pointer type, except there we bypassed visit_type_FOO().
But for simple unions where the current branch had scalar type, the
contents of that scalar meant that the decision to call
visit_type_FOO() was data-dependent - the reason we got lucky there
is that visit_type_FOO() for all scalar types in the dealloc visitor
is a no-op (only the pointer variants had anything to free), so it
did not matter whether the dealloc visit was skipped. But with this
patch, we would risk leaking memory if we could skip a call to
visit_type_FOO_fields() based solely on a data-dependent decision.

But notice: in the dealloc visitor, visit_type_FOO() already handles
a NULL obj - it was only the visit_type_implicit_FOO() that was
failing to check for NULL. And now that we have refactored things to
have the branch be part of the parent struct, we no longer have a
separate pointer that can be NULL in the first place. So we can just
delete the call to visit_start_union() altogether, and blindly visit
the branch type; there is no change in behavior except to the dealloc
visitor, where we now unconditionally visit the branch, but where that
visit is now always safe (for a flat union, we can no longer
dereference NULL, and for a simple union, visit_type_FOO() was already
safely handling NULL on pointer types).

Unfortunately, simple unions are not as easy to switch to unboxed
layout; because we are special-casing the hidden implicit type with
a single 'data' member, we really DO need to keep calling another
layer of visit_start_struct(), with a second malloc; although there
are some cleanups planned for simple unions in later patches.

visit_start_union() and gen_visit_implicit_struct() are now unused.
Drop them.

Note that after this patch, the only remaining use of
visit_start_implicit_struct() is for alternate types; the next patch
will do further cleanup based on that fact.

Backports commit 544a3731591f5d53e15f22de00ce5ac758d490b3 from qemu
2018-02-20 16:44:55 -05:00
Alistair Francis
a4bf026460
qom: Correct object_property_get_int() description
The description of object_property_get_int() stated that on an error
it returns NULL. This is not the case and the function will return -1
if an error occurs. Update the commented documentation accordingly.

Backports commit b29b47e9b35017428904e0e934700877dfaabe73 from qemu
2018-02-20 11:52:16 -05:00
Sergey Fedorov
6a3038db7c
cpu: Add callback to check architectural watchpoint match
When QEMU watchpoint matches, that is not definitely an architectural
watchpoint match yet. If it is a stop-before-access watchpoint then that
is hardly possible to ignore it after throwing a TCG exception.

A special callback is introduced to check for architectural watchpoint
match before raising a TCG exception.

Backports commit 568496c0c0f1863a4bc18539962cd8d81baa4e30 from qemu
2018-02-20 11:43:56 -05:00
Lioncash
c658126845
include: Move RAMList to ramlist.h
Moves the struct back into qemu's headers
2018-02-20 08:47:51 -05:00
Lioncash
cdd4003ce9
Move RAMBlock to ram_addr.h
Moves it back into qemu's includes.
2018-02-20 08:35:44 -05:00
Paolo Bonzini
cbc56b3ceb
memory: add early bail out from cpu_physical_memory_set_dirty_range
This condition is true in the common case, so we can cut out the body of
the function. In addition, this makes it easier for the compiler to do
at least partial inlining, even if it decides that fully inlining the
function is unreasonable.

Backports commit 8bafcb21643a39a5b29109f8bd5ee5a6f0f6850b from qemu
2018-02-20 08:32:10 -05:00
Lioncash
a268815478
include: Add stubbed xen function
Will allow us to not comment out code all the time for xen checks (ideally)
2018-02-20 08:29:58 -05:00
Markus Armbruster
91b4e76fef
error: New error_fatal
Similar to error_abort, but doesn't report where the error was
created, and terminates the process with exit(1) rather than abort().

Backports commit a29a37b994ca3c5a1d39fa0e8934f7e0f2cf57ef from qemu
2018-02-20 08:22:27 -05:00
Markus Armbruster
c4a06e2d12
error: Improve documentation some more
Don't claim error_report_err() always reports to stderr. It actually
reports to the current monitor when we have one.

Clarify intended use of error_abort and error_fatal.

Backports commit 10303f04b98efa76e638b9ae4632688f56f088fc from qemu
2018-02-20 08:13:39 -05:00
Eric Blake
1f54314cbb
qapi: Drop unused 'kind' for struct/enum visit
visit_start_struct() and visit_type_enum() had a 'kind' argument
that was usually set to either the stringized version of the
corresponding qapi type name, or to NULL (although some clients
didn't even get that right). But nothing ever used the argument.
It's even hard to argue that it would be useful in a debugger,
as a stack backtrace also tells which type is being visited.

Therefore, drop the 'kind' argument as dead.

Backports commit 337283dffbb5ad5860ed00408a5fd0665c21be07 from qemu
2018-02-19 23:43:54 -05:00
Eric Blake
844c136945
qapi: Swap 'name' in visit_* callbacks to match public API
As explained in the previous patches, matching argument order of
'name, &value' to JSON's "name":value makes sense. However,
while the last two patches were easy with Coccinelle, I ended up
doing this one all by hand. Now all the visitor callbacks match
the main interface.

The compiler is able to enforce that all clients match the changed
interface in visitor-impl.h, even where two pointers are being
swapped, because only one of the two pointers is const (if that
were not the case, then C's looseness on treating 'char *' like
'void *' would have made review a bit harder).

Backports commit 0b2a0d6bb2446060944061e53e87d0c7addede79 from qemu
2018-02-19 23:36:52 -05:00
Eric Blake
4100f3b78a
qapi: Consolidate visitor small integer callbacks
Commit 4e27e819 introduced optional visitor callbacks for all
sorts of int types, but no visitor has supplied any of the
callbacks for sizes less than 64 bits. In other words, the
generic implementation based on using type_[u]int64() followed
by bounds-checking works just fine. In the interest of
simplicity, it's easier to make the visitor callback interface
not have to worry about the other sizes.

Adding some helper functions minimizes the boilerplate required
to correct FIXMEs added earlier with regards to questionable
reuse of errp, particularly now that we can guarantee from a
single file audit that value is unchanged if an error is set.

Backports commit 04e070d217b4414f1f91aa8ad25fc0ae7ca0be93 from qemu
2018-02-19 23:21:56 -05:00
Eric Blake
9ec25b4673
qom: Swap 'name' next to visitor in ObjectPropertyAccessor
Similar to the previous patch, it's nice to have all functions
in the tree that involve a visitor and a name for conversion to
or from QAPI to consistently stick the 'name' parameter next
to the Visitor parameter.

Done by manually changing include/qom/object.h and qom/object.c,
then running this Coccinelle script and touching up the fallout
(Coccinelle insisted on adding some trailing whitespace).

@ rule1 @
identifier fn;
typedef Object, Visitor, Error;
identifier obj, v, opaque, name, errp;
@@
void fn
- (Object *obj, Visitor *v, void *opaque, const char *name,
+ (Object *obj, Visitor *v, const char *name, void *opaque,
Error **errp) { ... }

@@
identifier rule1.fn;
expression obj, v, opaque, name, errp;
@@
fn(obj, v,
- opaque, name,
+ name, opaque,
errp)

Backports commit d7bce9999df85c56c8cb1fcffd944d51bff8ff48 from qemu
2018-02-19 23:14:37 -05:00
Eric Blake
5dd5646a9a
qapi: Swap visit_* arguments for consistent 'name' placement
JSON uses "name":value, but many of our visitor interfaces were
called with visit_type_FOO(v, &value, name, errp). This can be
a bit confusing to have to mentally swap the parameter order to
match JSON order. It's particularly bad for visit_start_struct(),
where the 'name' parameter is smack in the middle of the
otherwise-related group of 'obj, kind, size' parameters! It's
time to do a global swap of the parameter ordering, so that the
'name' parameter is always immediately after the Visitor argument.

Additional reason in favor of the swap: the existing include/qjson.h
prefers listing 'name' first in json_prop_*(), and I have plans to
unify that file with the qapi visitors; listing 'name' first in
qapi will minimize churn to the (admittedly few) qjson.h clients.

Later patches will then fix docs, object.h, visitor-impl.h, and
those clients to match.

Done by first patching scripts/qapi*.py by hand to make generated
files do what I want, then by running the following Coccinelle
script to affect the rest of the code base:
$ spatch --sp-file script `git grep -l '\bvisit_' -- '**/*.[ch]'`
I then had to apply some touchups (Coccinelle insisted on TAB
indentation in visitor.h, and botched the signature of
visit_type_enum() by rewriting 'const char *const strings[]' to
the syntactically invalid 'const char*const[] strings'). The
movement of parameters is sufficient to provoke compiler errors
if any callers were missed.

// Part 1: Swap declaration order
@@
type TV, TErr, TObj, T1, T2;
identifier OBJ, ARG1, ARG2;
@@
void visit_start_struct
-(TV v, TObj OBJ, T1 ARG1, const char *name, T2 ARG2, TErr errp)
+(TV v, const char *name, TObj OBJ, T1 ARG1, T2 ARG2, TErr errp)
{ ... }

@@
type bool, TV, T1;
identifier ARG1;
@@
bool visit_optional
-(TV v, T1 ARG1, const char *name)
+(TV v, const char *name, T1 ARG1)
{ ... }

@@
type TV, TErr, TObj, T1;
identifier OBJ, ARG1;
@@
void visit_get_next_type
-(TV v, TObj OBJ, T1 ARG1, const char *name, TErr errp)
+(TV v, const char *name, TObj OBJ, T1 ARG1, TErr errp)
{ ... }

@@
type TV, TErr, TObj, T1, T2;
identifier OBJ, ARG1, ARG2;
@@
void visit_type_enum
-(TV v, TObj OBJ, T1 ARG1, T2 ARG2, const char *name, TErr errp)
+(TV v, const char *name, TObj OBJ, T1 ARG1, T2 ARG2, TErr errp)
{ ... }

@@
type TV, TErr, TObj;
identifier OBJ;
identifier VISIT_TYPE =~ "^visit_type_";
@@
void VISIT_TYPE
-(TV v, TObj OBJ, const char *name, TErr errp)
+(TV v, const char *name, TObj OBJ, TErr errp)
{ ... }

// Part 2: swap caller order
@@
expression V, NAME, OBJ, ARG1, ARG2, ERR;
identifier VISIT_TYPE =~ "^visit_type_";
@@
(
-visit_start_struct(V, OBJ, ARG1, NAME, ARG2, ERR)
+visit_start_struct(V, NAME, OBJ, ARG1, ARG2, ERR)
|
-visit_optional(V, ARG1, NAME)
+visit_optional(V, NAME, ARG1)
|
-visit_get_next_type(V, OBJ, ARG1, NAME, ERR)
+visit_get_next_type(V, NAME, OBJ, ARG1, ERR)
|
-visit_type_enum(V, OBJ, ARG1, ARG2, NAME, ERR)
+visit_type_enum(V, NAME, OBJ, ARG1, ARG2, ERR)
|
-VISIT_TYPE(V, OBJ, NAME, ERR)
+VISIT_TYPE(V, NAME, OBJ, ERR)
)

Backports commit 51e72bc1dd6ace6e91d675f41a1f09bd00ab8043 from qemu
2018-02-19 22:45:07 -05:00
Eric Blake
5b6f0cbdb7
qom: Use typedef for Visitor
No need to repeat 'struct Visitor' when we already have it in
typedefs.h. Omitting the redundant 'struct' also makes a later
patch easier to search for all object property callbacks that
are associated with a Visitor.

Backports commit 4fa45492c3387c0fa51e8e81160ac9a7814f44a2 from qemu
2018-02-19 22:26:47 -05:00
Eric Blake
7e83274012
qapi-visit: Kill unused visit_end_union()
The generated code can call visit_end_union() without having called
visit_start_union(). Example:

if (!*obj) {
goto out_obj;
}
visit_type_CpuInfoBase_fields(v, (CpuInfoBase **)obj, &err);
if (err) {
goto out_obj; // if we go from here...
}
if (!visit_start_union(v, !!(*obj)->u.data, &err) || err) {
goto out_obj;
}
switch ((*obj)->arch) {
[...]
}
out_obj:
// ... then *obj is true, and ...
error_propagate(errp, err);
err = NULL;
if (*obj) {
// we end up here
visit_end_union(v, !!(*obj)->u.data, &err);
}
error_propagate(errp, err);

Harmless only because no visitor implements end_union(). Clean it up
anyway, by deleting the function as useless.

Messed up since we have visit_end_union (commit cee2ded).

Backports commit 7c91aabd8964cfdf637f302c579c95401f21ce92 from qemu
2018-02-19 22:22:24 -05:00
Eric Blake
994490d197
qapi: Shorter visits of optional fields
For less code, reflect the determined boolean value of an optional
visit back to the caller instead of making the caller read the
boolean after the fact.

The resulting generated code has the following diff:

|- visit_optional(v, &has_fdset_id, "fdset-id");
|- if (has_fdset_id) {
|+ if (visit_optional(v, &has_fdset_id, "fdset-id")) {
| visit_type_int(v, &fdset_id, "fdset-id", &err);
| if (err) {
| goto out;
| }
| }

Backports commit 29637a6ee913df8fcdf371426ee48956b945b618 from qemu
2018-02-19 22:03:23 -05:00
Eric Blake
f3d2380f6d
qapi: Simplify visits of optional fields
None of the visitor callbacks would set an error when testing
if an optional field was present; make this part of the interface
contract by eliminating the errp argument.

The resulting generated code has a nice diff:

|- visit_optional(v, &has_fdset_id, "fdset-id", &err);
|- if (err) {
|- goto out;
|- }
|+ visit_optional(v, &has_fdset_id, "fdset-id");
| if (has_fdset_id) {
| visit_type_int(v, &fdset_id, "fdset-id", &err);
| if (err) {
| goto out;
| }
| }

Backports commit 5cdc8831a795fb8452d7e34f644202fd724e122a from qemu
2018-02-19 22:01:27 -05:00
Eric Blake
65d58b543e
qapi: Fix alternates that accept 'number' but not 'int'
The QMP input visitor allows integral values to be assigned by
promotion to a QTYPE_QFLOAT. However, when parsing an alternate,
we did not take this into account, such that an alternate that
accepts 'number' and some other type, but not 'int', would reject
integral values.

With this patch, we now have the following desirable table:

alternate has case selected for
'int' 'number' QTYPE_QINT QTYPE_QFLOAT
no no error error
no yes 'number' 'number'
yes no 'int' error
yes yes 'int' 'number'

While it is unlikely that we will ever use 'number' in an
alternate other than in the testsuite, it never hurts to be
more precise in what we allow.

Backports commit d00341af384665d259af475b14c96bb8414df415 from qemu
2018-02-19 21:58:10 -05:00
Eric Blake
2ee6c960ee
qapi: Simplify visiting of alternate types
Previously, working with alternates required two lookup arrays
and some indirection: for type Foo, we created Foo_qtypes[]
which maps each qtype to a value of the generated FooKind enum,
then look up that value in FooKind_lookup[] like we do for other
union types.

This has a couple of subtle bugs. First, the generator was
creating a call with a parameter '(int *) &(*obj)->type' where
type is an enum type; this is unsafe if the compiler chooses
to store the enum type in a different size than int, where
assigning through the wrong size pointer can corrupt data or
cause a SIGBUS.

Related bug, not not fixed in this patch: qapi-visit.py's
gen_visit_enum() generates a cast of its enum * argument to
int *. Marked FIXME.

Second, since the values of the FooKind enum start at zero, all
entries of the Foo_qtypes[] array that were not explicitly
initialized will map to the same branch of the union as the
first member of the alternate, rather than triggering a desired
failure in visit_get_next_type(). Fortunately, the bug seldom
bites; the very next thing the input visitor does is try to
parse the incoming JSON with the wrong parser, which normally
fails; the output visitor is not used with a C struct in that
state, and the dealloc visitor has nothing to clean up (so
there is no leak).

However, the second bug IS observable in one case: parsing an
integer causes unusual behavior in an alternate that contains
at least a 'number' member but no 'int' member, because the
'number' parser accepts QTYPE_QINT in addition to the expected
QTYPE_QFLOAT (that is, since 'int' is not a member, the type
QTYPE_QINT accidentally maps to FooKind 0; if this enum value
is the 'number' branch the integer parses successfully, but if
the 'number' branch is not first, some other branch tries to
parse the integer and rejects it). A later patch will worry
about fixing alternates to always parse all inputs that a
non-alternate 'number' would accept, for now this is still
marked FIXME in the updated test-qmp-input-visitor.c, to
merely point out that new undesired behavior of 'ans' matches
the existing undesired behavior of 'asn'.

This patch fixes the default-initialization bug by deleting the
indirection, and modifying get_next_type() to directly assign a
QTypeCode parameter. This in turn fixes the type-casting bug,
as we are no longer casting a pointer to enum to a questionable
size. There is no longer a need to generate an implicit FooKind
enum associated with the alternate type (since the QMP wire
format never uses the stringized counterparts of the C union
member names). Since the updated visit_get_next_type() does not
know which qtypes are expected, the generated visitor is
modified to generate an error statement if an unexpected type is
encountered.

Callers now have to know the QTYPE_* mapping when looking at the
discriminator; but so far, only the testsuite was even using the
C struct of an alternate types. I considered the possibility of
keeping the internal enum FooKind, but initialized differently
than most generated arrays, as in:
typedef enum FooKind {
FOO_KIND_A = QTYPE_QDICT,
FOO_KIND_B = QTYPE_QINT,
} FooKind;
to create nicer aliases for knowing when to use foo->a or foo->b
when inspecting foo->type; but it turned out to add too much
complexity, especially without a client.

There is a user-visible side effect to this change, but I
consider it to be an improvement. Previously,
the invalid QMP command:
{"execute":"blockdev-add", "arguments":{"options":
{"driver":"raw", "id":"a", "file":true}}}
failed with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: QDict"}}
(visit_get_next_type() succeeded, and the error comes from the
visit_type_BlockdevOptions() expecting {}; there is no mention of
the fact that a string would also work). Now it fails with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: BlockdevRef"}}
(the error when the next type doesn't match any expected types for
the overall alternate).

Backports commit 0426d53c6530606bf7641b83f2b755fe61c280ee from qemu
2018-02-19 21:52:39 -05:00
Eric Blake
e9666e4455
qapi: Convert QType into QAPI built-in enum type
What's more meta than using qapi to define qapi? :)

Convert QType into a full-fledged[*] builtin qapi enum type, so
that a subsequent patch can then use it as the discriminator
type of qapi alternate types. Fortunately, the judicious use of
'prefix' in the qapi definition avoids churn to the spelling of
the enum constants.

To avoid circular definitions, we have to flip the order of
inclusion between "qobject.h" vs. "qapi-types.h". Back in commit
28770e0, we had the latter include the former, so that we could
use 'QObject *' for our implementation of 'any'. But that usage
also works with only a forward declaration, whereas the
definition of QObject requires QType to be a complete type.

[*] The type has to be builtin, rather than declared in
qapi/common.json, because we want to use it for alternates even
when common.json is not included. But since it is the first
builtin enum type, we have to add special cases to qapi-types
and qapi-visit to only emit definitions once, even when two
qapi files are being compiled into the same binary (the way we
already handled builtin list types like 'intList'). We may
need to revisit how multiple qapi files share common types,
but that's a project for another day.

Backports commit 7264f5c50cc1be0f1406e3ebb45aedcca02f603a from qemu
2018-02-19 21:47:05 -05:00
Eric Blake
805c803298
qobject: Rename qtype_code to QType
The name QType matches our CODING_STYLE conventions for type names
in CamelCase. It also matches the fact that we are already naming
all the enum members with a prefix of QTYPE, not QTYPE_CODE. And
doing the rename will also make it easier for the next patch to use
QAPI for providing the enum, which also wants CamelCase type names.

Backports commit 1310a3d3bd9301ff5a825287638cfab24c2c6689 from qemu
2018-02-19 21:41:52 -05:00
Eric Blake
cc1d62568e
qobject: Simplify QObject
The QObject hierarchy is small enough, and unlikely to grow further
(since we only use it to map to JSON and already cover all JSON
types), that we can simplify things by not tracking a separate
vtable, but just inline the code element of the vtable QType
directly into QObject (renamed to type), and track a separate array
of destroy functions. We can drop qnull_destroy_obj() in the
process.

The remaining QObject subclasses must export their destructor.

This also has the nice benefit of moving the typename 'QType'
out of the way, so that the next patch can repurpose it for a
nicer name for 'qtype_code'.

The various objects are still the same size (so no change in cache
line pressure), but now have less indirection (although I didn't
bother benchmarking to see if there is a noticeable speedup, as
we don't have hard evidence that this was in a performance hotspot
in the first place).

A future patch could drop the refcnt size to 32 bits for a smaller
struct on 64-bit architectures, if desired (we have limits on the
largest JSON that we are willing to parse, and will probably never
need to take full advantage of a 64-bit refcnt).

Backports commit 55e1819c509b3d9c10a54678b9c585bbda13889e from qemu
2018-02-19 21:37:48 -05:00
Markus Armbruster
105a6be9b0
qobject: Add a special null QObject
I'm going to fix the JSON parser to recognize null. The obvious
representation of JSON null as (QObject *)NULL doesn't work, because
the parser already uses it as an error value. Perhaps we should
change it to free NULL for null, but that's more than I can do right
now. Create a special null QObject instead.

The existing QDict, QList, and QString all represent something that
is a pointer in C and could therefore be associated with NULL. But
right now, all three of these sub-types are always non-null once
created, so the new null sentinel object is intentionally unrelated
to them.

Backports commit 481b002cc81ed7fc7b06e32e9d4d495d81739d14 from qemu
2018-02-19 21:25:58 -05:00
Eric Blake
0ae71ac202
qapi: Change munging of CamelCase enum values
When munging enum values, the fact that we were passing the entire
prefix + value through camel_to_upper() meant that enum values
spelled with CamelCase could be turned into CAMEL_CASE. However,
this provides a potential collision (both OneTwo and One-Two would
munge into ONE_TWO) for enum types, when the same two names are
valid side-by-side as QAPI member names. By changing the generation
of enum constants to always be prefix + '_' + c_name(value,
False).upper(), and ensuring that there are no case collisions (in
the next patches), we no longer have to worry about names that
would be distinct as QAPI members but collide as variant tag names,
without having to think about what munging the heuristics in
camel_to_upper() will actually perform on an enum value.

Making the change will affect enums that did not follow coding
conventions, using 'CamelCase' rather than desired 'lower-case'.

Thankfully, there are only two culprits: InputButton and ErrorClass.
We already tweaked ErrorClass to make it an alias of QapiErrorClass,
where only the alias needs changing rather than the whole tree. So
the bulk of this change is modifying INPUT_BUTTON_WHEEL_UP to the
new INPUT_BUTTON_WHEELUP (and likewise for WHEELDOWN). That part
of this commit may later need reverting if we rename the enum
constants from 'WheelUp' to 'wheel-up' as part of moving
x-input-send-event to a stable interface; but at least we have
documentation bread crumbs in place to remind us (commit 513e7cd),
and it matches the fact that SDL constants are also spelled
SDL_BUTTON_WHEELUP.

Backports commit d20a580bc0eac9d489884f6d2ed28105880532b6 from qemu
2018-02-19 20:40:15 -05:00
Eric Blake
1bda2b186b
qapi: Add alias for ErrorClass
The qapi enum ErrorClass is unusual that it uses 'CamelCase' names,
contrary to our documented convention of preferring 'lower-case'.
However, this enum is entrenched in the API; we cannot change
what strings QMP outputs. Meanwhile, we want to simplify how
c_enum_const() is used to generate enum constants, by moving away
from the heuristics of camel_to_upper() to a more straightforward
c_name(N).upper() - but doing so will rename all of the ErrorClass
constants and cause churn to all client files, where the new names
are aesthetically less pleasing (ERROR_CLASS_DEVICENOTFOUND looks
like we can't make up our minds on whether to break between words).

So as always in computer science, solve the problem by some more
indirection: rename the qapi type to QapiErrorClass, and add a
new enum ErrorClass in error.h whose members are aliases of the
qapi type, but with the spelling expected elsewhere in the tree.
Then, when c_enum_const() changes the munging, we only have to
adjust the one alias spot.

Backports commit f22a28b898322c01b0463a8b7ec551d72bc61a5b from qemu
2018-02-19 20:38:51 -05:00
Eric Blake
a5cbe099d7
qapi: Remove dead visitor code
Commit cbc95538 removed unused start_handle() and end_handle(),
but forgot to remove their declarations.

Backports commit 7549457200ec3871ee827765f4d3bbc8d903b2ec from qemu
2018-02-19 20:32:00 -05:00
Markus Armbruster
f93438ba43
qapi: Introduce a first class 'any' type
It's first class, because unlike '**', it actually works, i.e. doesn't
require 'gen': false.

'**' will go away next.

Backports commit 28770e057f265a4e70bcbdfc2447cce7b5f2dc19 from qemu
2018-02-19 17:46:58 -05:00
Daniel P. Berrange
767e900547
qom: Make enum string tables const-correct
The enum string table parameters in various QOM/QAPI methods
are declared 'const char *strings[]'. This results in const
warnings if passed a variable that was declared as

   static const char * const strings[] = { .... };

Add the extra const annotation to the parameters, since
neither the string elements, nor the array itself should
ever be modified.

Backports commit 2e4450ff432daef524cb3557fca68a3b7b5c7823 from qemu
2018-02-19 16:02:23 -05:00
Eric Blake
3aba81d5aa
qapi: Drop unused error argument for list and implicit struct
No backend was setting an error when ending the visit of a list or
implicit struct, or when moving to the next list node. Make the
callers a bit easier to follow by making this a part of the contract,
and removing the errp argument - callers can then unconditionally end
an object as part of cleanup without having to think about whether a
second error is dominated by a first, because there is no second
error.

A later patch will then tackle the larger task of splitting
visit_end_struct(), which can indeed set an error.

Backports commit 08f9541dec51700abef0c37994213164ca4e4fc9 from qemu
2018-02-19 12:59:54 -05:00
Eric Blake
eeffd97458
qapi: Make all visitors supply uint64 callbacks
Our qapi visitor contract supports multiple integer visitors,
but left the type_uint64 visitor as optional (falling back on
type_int64); which in turn can lead to awkward behavior with
numbers larger than INT64_MAX (the user has to be aware of
twos complement, and deal with negatives).

This patch does not address the disparity in handling large
values as negatives. It merely moves the fallback from uint64
to int64 from the visitor core to the visitors, where the issue
can actually be fixed, by implementing the missing type_uint64()
callbacks on top of the respective type_int64() callbacks, and
with a FIXME comment explaining why that's wrong.

With that done, we now have a type_uint64() callback in every
driver, so we can make it mandatory from the core. And although
the type_int64() callback can cover the entire valid range of
type_uint{8,16,32} on valid user input, using type_uint64() to
avoid mixed signedness makes more sense.

Backports commit f755dea79dc81b0d6a8f6414e0672e165e28d8ba from qemu
2018-02-19 11:59:22 -05:00
Eric Blake
5b5299bdee
qapi: Prefer type_int64 over type_int in visitors
The qapi builtin type 'int' is basically shorthand for the type
'int64'. In fact, since no visitor was providing the optional
type_int64() callback, visit_type_int64() was just always falling
back to type_int(), cementing the equivalence between the types.

However, some visitors are providing a type_uint64() callback.
For purposes of code consistency, it is nicer if all visitors
use the paired type_int64/type_uint64 names rather than the
mismatched type_int/type_uint64. So this patch just renames
the signed int callbacks in place, dropping the type_int()
callback as redundant, and a later patch will focus on the
unsigned int callbacks.

Add some FIXMEs to questionable reuse of errp in code touched
by the rename, while at it (the reuse works as long as the
callbacks don't modify value when setting an error, but it's not
a good example to set) - a later patch will then fix those.

No change in functionality here, although further cleanups are
in the pipeline.

Backports commit 4c40314a35816de635e7170eaacdc0c35be83a8a from qemu
2018-02-19 11:53:21 -05:00
Peter Maydell
c8220d5aaf
fpu: Replace uint8 typedef with uint8_t
Replace the uint8 softfloat-specific typedef with uint8_t.
This change was made with

find include hw fpu target-* -name '*.[ch]' | xargs sed -i -e 's/\buint8\b/uint8_t/g'

together with manual removal of the typedef definition and
manual fixing of more erroneous uses found via test compilation.

It turns out that the only code using this type is an accidental
use where uint8_t was intended anyway...

Backports commit d341d9f3062c74d74c94ebe6359f067bed8311ba from qemu
2018-02-19 00:34:41 -05:00
Peter Maydell
2eaf79bfd3
fpu: Replace int8 typedef with int8_t
Replace the int8 softfloat-specific typedef with int8_t.
This change was made with

find include hw fpu target-* -name '*.[ch]' | xargs sed -i -e 's/\bint8\b/int8_t/g'

together with manual removal of the typedef definition, and
manual undoing of various mis-hits.

Backports commit 8f506c709adb7d3bed4ebefefe9487c156192a64 from qemu
2018-02-19 00:33:01 -05:00
Peter Maydell
63ac282f1a
fpu: Replace uint32 typedef with uint32_t
Replace the uint32 softfloat-specific typedef with uint32_t.
This change was made with

find include hw fpu target-* -name '*.[ch]' | xargs sed -i -e 's/\buint32\b/uint32_t/g'

together with manual removal of the typedef definition,
manual undoing of various mis-hits, and another couple of
fixes found via test compilation.

All the uses in hw/ were using the wrong type by mistake.

Backports commit 3a87d00910ef64a2eece4aad25d96ea10683fc5c from qemu
2018-02-19 00:29:52 -05:00
Peter Maydell
9712d8a7ac
fpu: Replace int32 typedef with int32_t
Replace the int32 softfloat-specific typedef with int32_t.
This change was made with

find hw include fpu target-* -name '*.[ch]' | xargs sed -i -e 's/\bint32\b/int32_t/g'

together with manual removal of the typedef definition, and
manual undoing of some mis-hits where macro arguments were
being used for token pasting rather than as a type.

The uses in hw/ipmi/ should not have been using this type at all.

Backports commit f4014512cda682a9d0c75310d278d7ae96b0505c from qemu
2018-02-19 00:24:56 -05:00
Peter Maydell
df84f0d513
fpu: Replace uint64 typedef with uint64_t
Replace the uint64 softfloat-specific typedef with uint64_t.
This change was made with

find include fpu target-* -name '*.[ch]' | xargs sed -i -e 's/\buint64\b/uint64_t/g'

together with manual removal of the typedef definition, and
manual undoing of some mis-hits where macro arguments were
being used for token pasting rather than as a type.

Note that the target-mips/kvm.c and target-s390x/kvm.c changes are fixing
code that should not have been using the uint64 type in the first place.

Backports commit 182f42fdc219e6481654fcfb73b17e4b4e63b6ff from qemu
2018-02-19 00:16:49 -05:00
Peter Maydell
940106cd69
fpu: Replace int64 typedef with int64_t
Replace the int64 softfloat-specific typedef with int64_t.
This change was made with

find include fpu target-* -name '*.[ch]' | xargs sed -i -e 's/\bint64\b/int64_t/g'

together with manual removal of the typedef definition, and
manual undoing of some mis-hits where macro arguments were
being used for token pasting rather than as a type.

Backports commit f42c222482b651400f0fa417eb174da1c9502c1c from qemu
2018-02-19 00:14:18 -05:00
Peter Crosthwaite
ce997e1caf
qom/cpu: Add MemoryRegion property
Add a MemoryRegion property, which if set is used to construct
the CPU's initial (default) AddressSpace.

Backports commit 6731d864f80938e404dc3e5eb7f6b76b891e3e43 from qemu
2018-02-18 21:54:50 -05:00
Lioncash
6d5f465449
uc: Handle freeing of multiple address spaces 2018-02-18 21:36:50 -05:00
Dr. David Alan Gilbert
75701d03ee
qemu_ram_foreach_block: pass up error value, and down the ramblock name
check the return value of the function it calls and error if it's non-0
Fixup qemu_rdma_init_one_block that is the only current caller,
  and rdma_add_block the only function it calls using it.

Pass the name of the ramblock to the function; helps in debugging.

Backports commit e3807054e20fb3b94d18cb751c437ee2f43b6fac from qemu
2018-02-18 19:17:18 -05:00
Peter Crosthwaite
b82e711a65
memory: Add address_space_init_shareable()
This will either create a new AS or return a pointer to an
already existing equivalent one, if we have already created
an AS for the specified root memory region.

The motivation is to reuse address spaces as much as possible.
It's going to be quite common that bus masters out in device land
have pointers to the same memory region for their mastering yet
each will need to create its own address space. Let the memory
API implement sharing for them.

Aside from the perf optimisations, this should reduce the amount
of redundant output on info mtree as well.

Thee returned value will be malloced, but the malloc will be
automatically freed when the AS runs out of refs.

Backports commit f0c02d15b57da6f5463e3768aa0cfeedccf4b8f4 from qemu
2018-02-18 00:18:21 -05:00