In the near future, we want to lift our artificial restriction of
no variants at the top level of an event, at which point the
currently open-coded check for empty members will become
insufficient. Factor it out into a new helper method is_empty()
now, and future-proof it by checking variants, too, along with an
assert that it is not used prior to the completion of .check().
Update places that were checking for (non-)empty .members to use
the new helper.
All of the current callers assert that there are no variants (either
directly, or by qapi.py asserting that base types have no variants),
so this is not a semantic change.
No change to generated code.
Backports commit b6167706829c6e0d3572daa2b6769594ced276f7 from qemu
Making each visitor provide its own (awkwardly-named) FOO_cleanup()
is unusual, when we can instead have a polymorphic visit_free()
interface. Over the next few patches, we can use the polymorphic
functions to eliminate the need for a FOO_get_visitor() function
for accessing specific visitor functionality, once everything can
be accessed directly through the Visitor* interfaces.
The dealloc visitor is the first one converted to completely use
the new entry point, since qapi_dealloc_visitor_cleanup() was the
only reason that qapi_dealloc_get_visitor() existed, and only
generated and testsuite code was even using it. With the new
visit_free() entry point in place, we no longer need to expose
the QapiDeallocVisitor subtype through qapi_dealloc_visitor_new(),
and can get by with less generated code, with diffs that look like:
| void qapi_free_ACPIOSTInfo(ACPIOSTInfo *obj)
| {
|- QapiDeallocVisitor *qdv;
| Visitor *v;
|
| if (!obj) {
| return;
| }
|
|- qdv = qapi_dealloc_visitor_new();
|- v = qapi_dealloc_get_visitor(qdv);
|+ v = qapi_dealloc_visitor_new();
| visit_type_ACPIOSTInfo(v, NULL, &obj, NULL);
|- qapi_dealloc_visitor_cleanup(qdv);
|+ visit_free(v);
|}
Backports commit 2c0ef9f411ae6081efa9eca5b3eab2dbeee45a6c from qemu
By sticking the next pointer first, we don't need a union with
64-bit padding for smaller types. On 32-bit platforms, this
can reduce the size of uint8List from 16 bytes (or 12, depending
on whether 64-bit ints can tolerate 4-byte alignment) down to 8.
It has no effect on 64-bit platforms (where alignment still
dictates a 16-byte struct); but fewer anonymous unions is still
a win in my book.
It requires visit_next_list() to gain a size parameter, to know
what size element to allocate; comparable to the size parameter
of visit_start_struct().
I debated about going one step further, to allow for fewer casts,
by doing:
typedef GenericList GenericList;
struct GenericList {
GenericList *next;
};
struct FooList {
GenericList base;
Foo *value;
};
so that you convert to 'GenericList *' by '&foolist->base', and
back by 'container_of(generic, GenericList, base)' (as opposed to
the existing '(GenericList *)foolist' and '(FooList *)generic').
But doing that would require hoisting the declaration of
GenericList prior to inclusion of qapi-types.h, rather than its
current spot in visitor.h; it also makes iteration a bit more
verbose through 'foolist->base.next' instead of 'foolist->next'.
Note that for lists of objects, the 'value' payload is still
hidden behind a boxed pointer. Someday, it would be nice to do:
struct FooList {
FooList *next;
Foo value;
};
for one less level of malloc for each list element. This patch
is a step in that direction (now that 'next' is no longer at a
fixed non-zero offset within the struct, we can store more than
just a pointer's-worth of data as the value payload), but the
actual conversion would be a task for another series, as it will
touch a lot of code.
Backports commit e65d89bf1a4484e0db0f3dc820a8b209f2fb1e8b from qemu
Rather than requiring all flat unions to explicitly create
a separate base struct, we can allow the qapi schema to specify
the common members via an inline dictionary. This is similar to
how commands can specify an inline anonymous type for its 'data'.
We already have several struct types that only exist to serve as
a single flat union's base; the next commit will clean them up.
In particular, this patch's change to the BlockdevOptions example
in qapi-code-gen.txt will actually be done in the real QAPI schema.
Now that anonymous bases are legal, we need to rework the
flat-union-bad-base negative test (as previously written, it
forms what is now valid QAPI; tweak it to now provide coverage
of a new error message path), and add a positive test in
qapi-schema-test to use an anonymous base (making the integer
argument optional, for even more coverage).
Note that this patch only allows anonymous bases for flat unions;
simple unions are already enough syntactic sugar that we do not
want to burden them further. Meanwhile, while it would be easy
to also allow an anonymous base for structs, that would be quite
redundant, as the members can be put right into the struct
instead.
Backports commit ac4338f8eb783fd421aae492ca262a586918471e from qemu
Simple unions were carrying a special case that hid their 'data'
QMP member from the resulting C struct, via the hack method
QAPISchemaObjectTypeVariant.simple_union_type(). But by using
the work we started by unboxing flat union and alternate
branches, coupled with the ability to visit the members of an
implicit type, we can now expose the simple union's implicit
type in qapi-types.h:
| struct q_obj_ImageInfoSpecificQCow2_wrapper {
| ImageInfoSpecificQCow2 *data;
| };
|
| struct q_obj_ImageInfoSpecificVmdk_wrapper {
| ImageInfoSpecificVmdk *data;
| };
...
| struct ImageInfoSpecific {
| ImageInfoSpecificKind type;
| union { /* union tag is @type */
| void *data;
|- ImageInfoSpecificQCow2 *qcow2;
|- ImageInfoSpecificVmdk *vmdk;
|+ q_obj_ImageInfoSpecificQCow2_wrapper qcow2;
|+ q_obj_ImageInfoSpecificVmdk_wrapper vmdk;
| } u;
| };
Doing this removes asymmetry between QAPI's QMP side and its
C side (both sides now expose 'data'), and means that the
treatment of a simple union as sugar for a flat union is now
equivalent in both languages (previously the two approaches used
a different layer of dereferencing, where the simple union could
be converted to a flat union with equivalent C layout but
different {} on the wire, or to an equivalent QMP wire form
but with different C representation). Using the implicit type
also lets us get rid of the simple_union_type() hack.
Of course, now all clients of simple unions have to adjust from
using su->u.member to using su->u.member.data; while this touches
a number of files in the tree, some earlier cleanup patches
helped minimize the change to the initialization of a temporary
variable rather than every single member access. The generated
qapi-visit.c code is also affected by the layout change:
|@@ -7393,10 +7393,10 @@ void visit_type_ImageInfoSpecific_member
| }
| switch (obj->type) {
| case IMAGE_INFO_SPECIFIC_KIND_QCOW2:
|- visit_type_ImageInfoSpecificQCow2(v, "data", &obj->u.qcow2, &err);
|+ visit_type_q_obj_ImageInfoSpecificQCow2_wrapper_members(v, &obj->u.qcow2, &err);
| break;
| case IMAGE_INFO_SPECIFIC_KIND_VMDK:
|- visit_type_ImageInfoSpecificVmdk(v, "data", &obj->u.vmdk, &err);
|+ visit_type_q_obj_ImageInfoSpecificVmdk_wrapper_members(v, &obj->u.vmdk, &err);
| break;
| default:
| abort();
Backports commit 32bafa8fdd098d52fbf1102d5a5e48d29398c0aa from qemu
We already have several places that want to visit all the members
of an implicit object within a larger context (simple union variant,
event with anonymous data, command with anonymous arguments struct);
and will be adding another one soon (the ability to declare an
anonymous base for a flat union). Having a C struct declared for
these implicit types, along with a visit_type_FOO_members() helper
function, will make for fewer special cases in our generator.
We do not, however, need qapi_free_FOO() or visit_type_FOO()
functions for implicit types, because they should not be used
directly outside of the generated code. This is done by adding a
conditional in visit_object_type() for both qapi-types.py and
qapi-visit.py based on the object name. The comparison of
"name.startswith('q_')" is a bit hacky (it's basically duplicating
what .is_implicit() already uses), but beats changing the signature
of the visit_object_type() callback to pass a new 'implicit' flag.
The hack should be temporary: we are considering adding a future
patch that consolidates the narrow visit_object_type(..., base,
local_members, variants) and visit_object_type_flat(...,
all_members, variants) [where different sets of information are
already broken out, and the QAPISchemaObjectType is no longer
available] into a broader visit_object_type(obj_type) [where the
visitor can query the needed fields from obj_type directly].
Also, now that we WANT to output C code for implicits, we no longer
need the visit_needed() filter, leaving 'q_empty' as the only object
still needing a special case. Remember, 'q_empty' is the only
built-in generated object, which means that without a special case
it would be emitted in multiple files (the main qapi-types.h and in
qga-qapi-types.h) causing compilation failure due to redefinition.
But since it has no members, it's easier to just avoid an attempt to
visit that particular type; since gen_object() is called recursively,
we also prime the objects_seen set to cover any recursion into the
empty type.
The patch relies on the changed naming of implicit types in the
previous patch. It is a bit unfortunate that the generated struct
names and visit_type_FOO_members() don't match normal naming
conventions, but it's not too bad, since they will only be used in
generated code.
The generated code grows substantially in size: the implicit
'-wrapper' types must be emitted in qapi-types.h before any union
can include an unboxed member of that type. Arguably, the '-args'
types could be emitted in a private header for just qapi-visit.c
and qmp-marshal.c, rather than polluting qapi-types.h; but adding
complexity to the generator to split the output location according
to role doesn't seem worth the maintenance costs.
Backports commit 7ce106a96feee4d46bfcdb47127b0935804c9357 from qemu
We started moving away from the use of the 'void *data' member
in the C union corresponding to a QAPI union back in commit
544a373; recent commits have gotten rid of other uses. Now
that it is completely unused, we can remove the member itself
as well as the FIXME comment. Update the testsuite to drop the
negative test union-clash-data.
Backports commit 48eb62a74fc2d6b0ae9e5f414304a85cfbf33066 from qemu
C types and JSON objects don't have fields, but members. We
shouldn't gratuitously invent terminology. This patch is a
strict renaming of static genarated functions, plus the naming
of the dummy filler member for empty structs, before the next
patch exposes some of that naming to the rest of the code base.
Backports commit c81200b01422783cd29796ef4ccc275d05f9ce67 from qemu
C types and JSON objects don't have fields, but members. We
shouldn't gratuitously invent terminology. This patch is a
strict renaming of generator code internals (including testsuite
comments), before later patches rename C interfaces.
No change to generated code with this patch.
Backports commit 14f00c6c492488381a513c3816b15794446231a0 from qemu
QAPISchemaType.c_type() is a bit awkward: it takes two optional
boolean flags is_param and is_unboxed, and they should never both
be True.
Add a new method for each of the flags, and drop the flags from
c_type().
Most callers pass no flags; they remain unchanged.
One caller passes is_param=True; call the new .c_param_type()
instead.
One caller passes is_unboxed=True, except for simple union types.
This is actually an ugly special case that will go away soon, so
until then, we now have to call either .c_type() or the new
.c_unboxed_type(). Tolerable in the interim.
It requires slightly more Python, but is arguably easier to read.
Backports commit 4040d995e49c5b818be79e50a18c1bf8d2354d12 from qemu
There's no reason to do two malloc's for a flat union; let's just
inline the branch struct directly into the C union branch of the
flat union.
Surprisingly, fewer clients were actually using explicit references
to the branch types in comparison to the number of flat unions
thus modified.
This lets us reduce the hack in qapi-types:gen_variants() added in
the previous patch; we no longer need to distinguish between
alternates and flat unions.
The change to unboxed structs means that u.data (added in commit
cee2dedb) is now coincident with random fields of each branch of
the flat union, whereas beforehand it was only coincident with
pointers (since all branches of a flat union have to be objects).
Note that this was already the case for simple unions - but there
we got lucky. Remember, visit_start_union() blindly returns true
for all visitors except for the dealloc visitor, where it returns
the value !!obj->u.data, and that this result then controls
whether to proceed with the visit to the variant. Pre-patch,
this meant that flat unions were testing whether the boxed pointer
was still NULL, and thereby skipping visit_end_implicit_struct()
and avoiding a NULL dereference if the pointer had not been
allocated. The same was true for simple unions where the current
branch had pointer type, except there we bypassed visit_type_FOO().
But for simple unions where the current branch had scalar type, the
contents of that scalar meant that the decision to call
visit_type_FOO() was data-dependent - the reason we got lucky there
is that visit_type_FOO() for all scalar types in the dealloc visitor
is a no-op (only the pointer variants had anything to free), so it
did not matter whether the dealloc visit was skipped. But with this
patch, we would risk leaking memory if we could skip a call to
visit_type_FOO_fields() based solely on a data-dependent decision.
But notice: in the dealloc visitor, visit_type_FOO() already handles
a NULL obj - it was only the visit_type_implicit_FOO() that was
failing to check for NULL. And now that we have refactored things to
have the branch be part of the parent struct, we no longer have a
separate pointer that can be NULL in the first place. So we can just
delete the call to visit_start_union() altogether, and blindly visit
the branch type; there is no change in behavior except to the dealloc
visitor, where we now unconditionally visit the branch, but where that
visit is now always safe (for a flat union, we can no longer
dereference NULL, and for a simple union, visit_type_FOO() was already
safely handling NULL on pointer types).
Unfortunately, simple unions are not as easy to switch to unboxed
layout; because we are special-casing the hidden implicit type with
a single 'data' member, we really DO need to keep calling another
layer of visit_start_struct(), with a second malloc; although there
are some cleanups planned for simple unions in later patches.
visit_start_union() and gen_visit_implicit_struct() are now unused.
Drop them.
Note that after this patch, the only remaining use of
visit_start_implicit_struct() is for alternate types; the next patch
will do further cleanup based on that fact.
Backports commit 544a3731591f5d53e15f22de00ce5ac758d490b3 from qemu
There's no reason to do two malloc's for an alternate type visiting
a QAPI struct; let's just inline the struct directly as the C union
branch of the struct.
Surprisingly, no clients were actually using the struct member prior
to this patch outside of the testsuite; an earlier patch in the series
added some testsuite coverage to make the effect of this patch more
obvious.
In qapi.py, c_type() gains a new is_unboxed flag to control when we
are emitting a C struct unboxed within the context of an outer
struct (different from our other two modes of usage with no flags
for normal local variable declarations, and with is_param for adding
'const' in a parameter list). I don't know if there is any more
pythonic way of collapsing the two flags into a single parameter,
as we never have a caller setting both flags at once.
Ultimately, we want to also unbox branches for QAPI unions, but as
that touches a lot more client code, it is better as separate
patches. But since unions and alternates share gen_variants(), I
had to hack in a way to test if we are visiting an alternate type
for setting the is_unboxed flag: look for a non-object branch.
This works because alternates have at least two branches, with at
most one object branch, while unions have only object branches.
The hack will go away in a later patch.
The generated code difference to qapi-types.h is relatively small:
| struct BlockdevRef {
| QType type;
| union { /* union tag is @type */
| void *data;
|- BlockdevOptions *definition;
|+ BlockdevOptions definition;
| char *reference;
| } u;
| };
The corresponding spot in qapi-visit.c calls visit_type_FOO(), which
first calls visit_start_struct() to allocate or deallocate the member
and handle a layer of {} from the JSON stream, then visits the
members. To peel off the indirection and the memory management that
comes with it, we inline this call, then suppress allocation /
deallocation by passing NULL to visit_start_struct(), and adjust the
member visit:
| switch ((*obj)->type) {
| case QTYPE_QDICT:
|- visit_type_BlockdevOptions(v, name, &(*obj)->u.definition, &err);
|+ visit_start_struct(v, name, NULL, 0, &err);
|+ if (err) {
|+ break;
|+ }
|+ visit_type_BlockdevOptions_fields(v, &(*obj)->u.definition, &err);
|+ error_propagate(errp, err);
|+ err = NULL;
|+ visit_end_struct(v, &err);
| break;
| case QTYPE_QSTRING:
| visit_type_str(v, name, &(*obj)->u.reference, &err);
The visit of non-object fields is unchanged.
Backports commit becceedc4d9bc1435099c90a0514945a89844d3a from qemu
Right now, we emit the branches of union types as a boxed pointer,
and it suffices to have a forward declaration of the type. However,
a future patch will swap things to directly use the branch type,
instead of hiding it behind a pointer. For this to work, the
compiler needs the full definition of the type, not just a forward
declaration, prior to the union that is including the branch type.
This patch just adds topological sorting to hoist all types
mentioned in a branch of a union to be fully declared before the
union itself. The sort is always possible, because we do not
allow circular union types that include themselves as a direct
branch (it is, however, still possible to include a branch type
that itself has a pointer to the union, for a type that can
indirectly recursively nest itself - that remains safe, because
that the member of the branch type will remain a pointer, and the
QMP representation of such a type adds another {} for each recurring
layer of the union type).
Backports commit 1de5d4ca0752138034305f3d4e8fe17ef6503569 from qemu
JSON uses "name":value, but many of our visitor interfaces were
called with visit_type_FOO(v, &value, name, errp). This can be
a bit confusing to have to mentally swap the parameter order to
match JSON order. It's particularly bad for visit_start_struct(),
where the 'name' parameter is smack in the middle of the
otherwise-related group of 'obj, kind, size' parameters! It's
time to do a global swap of the parameter ordering, so that the
'name' parameter is always immediately after the Visitor argument.
Additional reason in favor of the swap: the existing include/qjson.h
prefers listing 'name' first in json_prop_*(), and I have plans to
unify that file with the qapi visitors; listing 'name' first in
qapi will minimize churn to the (admittedly few) qjson.h clients.
Later patches will then fix docs, object.h, visitor-impl.h, and
those clients to match.
Done by first patching scripts/qapi*.py by hand to make generated
files do what I want, then by running the following Coccinelle
script to affect the rest of the code base:
$ spatch --sp-file script `git grep -l '\bvisit_' -- '**/*.[ch]'`
I then had to apply some touchups (Coccinelle insisted on TAB
indentation in visitor.h, and botched the signature of
visit_type_enum() by rewriting 'const char *const strings[]' to
the syntactically invalid 'const char*const[] strings'). The
movement of parameters is sufficient to provoke compiler errors
if any callers were missed.
// Part 1: Swap declaration order
@@
type TV, TErr, TObj, T1, T2;
identifier OBJ, ARG1, ARG2;
@@
void visit_start_struct
-(TV v, TObj OBJ, T1 ARG1, const char *name, T2 ARG2, TErr errp)
+(TV v, const char *name, TObj OBJ, T1 ARG1, T2 ARG2, TErr errp)
{ ... }
@@
type bool, TV, T1;
identifier ARG1;
@@
bool visit_optional
-(TV v, T1 ARG1, const char *name)
+(TV v, const char *name, T1 ARG1)
{ ... }
@@
type TV, TErr, TObj, T1;
identifier OBJ, ARG1;
@@
void visit_get_next_type
-(TV v, TObj OBJ, T1 ARG1, const char *name, TErr errp)
+(TV v, const char *name, TObj OBJ, T1 ARG1, TErr errp)
{ ... }
@@
type TV, TErr, TObj, T1, T2;
identifier OBJ, ARG1, ARG2;
@@
void visit_type_enum
-(TV v, TObj OBJ, T1 ARG1, T2 ARG2, const char *name, TErr errp)
+(TV v, const char *name, TObj OBJ, T1 ARG1, T2 ARG2, TErr errp)
{ ... }
@@
type TV, TErr, TObj;
identifier OBJ;
identifier VISIT_TYPE =~ "^visit_type_";
@@
void VISIT_TYPE
-(TV v, TObj OBJ, const char *name, TErr errp)
+(TV v, const char *name, TObj OBJ, TErr errp)
{ ... }
// Part 2: swap caller order
@@
expression V, NAME, OBJ, ARG1, ARG2, ERR;
identifier VISIT_TYPE =~ "^visit_type_";
@@
(
-visit_start_struct(V, OBJ, ARG1, NAME, ARG2, ERR)
+visit_start_struct(V, NAME, OBJ, ARG1, ARG2, ERR)
|
-visit_optional(V, ARG1, NAME)
+visit_optional(V, NAME, ARG1)
|
-visit_get_next_type(V, OBJ, ARG1, NAME, ERR)
+visit_get_next_type(V, NAME, OBJ, ARG1, ERR)
|
-visit_type_enum(V, OBJ, ARG1, ARG2, NAME, ERR)
+visit_type_enum(V, NAME, OBJ, ARG1, ARG2, ERR)
|
-VISIT_TYPE(V, OBJ, NAME, ERR)
+VISIT_TYPE(V, NAME, OBJ, ERR)
)
Backports commit 51e72bc1dd6ace6e91d675f41a1f09bd00ab8043 from qemu
Previously, the generated code in qapi-types.c initialized all
enum lookup tables first, prior to any other definitions. But
there are no topological sorting requirements that mandate this
layout, so we can drop the QAPISchemaGenTypeVisitor._fwdefn
field and just generate all definitions in visitation order.
The generated code shows some churn due to reordering, but it
is still fairly straightforward to follow (all the deletions
occur in one hunk, and all the deleted lines are re-inserted
in the same order later in the same files, just spread across
multiple insertion points).
Backports commit 0b2e84ba774651656771ed697dee8825759dffa9 from qemu
What's more meta than using qapi to define qapi? :)
Convert QType into a full-fledged[*] builtin qapi enum type, so
that a subsequent patch can then use it as the discriminator
type of qapi alternate types. Fortunately, the judicious use of
'prefix' in the qapi definition avoids churn to the spelling of
the enum constants.
To avoid circular definitions, we have to flip the order of
inclusion between "qobject.h" vs. "qapi-types.h". Back in commit
28770e0, we had the latter include the former, so that we could
use 'QObject *' for our implementation of 'any'. But that usage
also works with only a forward declaration, whereas the
definition of QObject requires QType to be a complete type.
[*] The type has to be builtin, rather than declared in
qapi/common.json, because we want to use it for alternates even
when common.json is not included. But since it is the first
builtin enum type, we have to add special cases to qapi-types
and qapi-visit to only emit definitions once, even when two
qapi files are being compiled into the same binary (the way we
already handled builtin list types like 'intList'). We may
need to revisit how multiple qapi files share common types,
but that's a project for another day.
Backports commit 7264f5c50cc1be0f1406e3ebb45aedcca02f603a from qemu
Simplify gen_struct_fields() back to a single iteration over a
list of fields (like it was prior to commit f87ab7f9), by moving
the generated comments to gen_object(). Then, inline
gen_struct_field() into its only caller.
Backports commit 7d9586f900a9043025866f84c096b1842b0bbbf6 from qemu
These two methods are now close enough that we can finally merge
them, relying on the fact that simple unions now provide a
reasonable local_members. Change gen_struct() to gen_object()
that handles all forms of QAPISchemaObjectType, and rename and
shrink gen_union() to gen_variants() to handle the portion of
gen_object() needed when variants are present.
gen_struct_fields() now has a single caller, so it no longer
needs an optional parameter; however, I did not choose to inline
it into the caller.
No difference to generated code.
Backports commit 570cd8d1194cf68f7c9948971e52e47f20855a77 from qemu
We were previously creating all unions with an empty list for
local_members. However, it will make it easier to unify struct
and union generation if we include the generated tag member in
local_members. That way, we can have a common code pattern:
visit the base (if any), visit the local members (if any), visit
the variants (if any). The local_members of a flat union
remains empty (because the discriminator is already visited as
part of the base). Then, by visiting tag_member.check() during
AlternateType.check(), we no longer need to call it during
Variants.check().
The various front end entities now exist as follows:
struct: optional base, optional local_members, no variants
simple union: no base, one-element local_members, variants with tag_member
from local_members
flat union: base, no local_members, variants with tag_member from base
alternate: no base, no local_members, variants
With the new local members, we require a bit of finesse to
avoid assertions in the clients.
No change to generated code.
Backports commit da34a9bd999d1be13954c699bbae0295cdaa3200 from qemu
Rather than having all callers pass a name, type, and optional
flag, have them instead pass a QAPISchemaObjectTypeMember which
already has all that information.
No change to generated code.
Backports commit 32bc6879beea0b0cac6196cb15a71d206401e96d from qemu
We have two issues with our qapi union layout:
1) Even though the QMP wire format spells the tag 'type', the
C code spells it 'kind', requiring some hacks in the generator.
2) The C struct uses an anonymous union, which places all tag
values in the same namespace as all non-variant members. This
leads to spurious collisions if a tag value matches a non-variant
member's name.
This patch is the back end for a series that converts to a
saner qapi union layout. Now that all clients have been
converted to use 'type' and 'obj->u.value', we can drop the
temporary parallel support for 'kind' and 'obj->value'.
Given a simple union qapi type:
{ 'union':'Foo', 'data': { 'a':'int', 'b':'bool' } }
this is the overall effect, when compared to the state before
this series of patches:
| struct Foo {
|- FooKind kind;
|- union { /* union tag is @kind */
|+ FooKind type;
|+ union { /* union tag is @type */
| void *data;
| int64_t a;
| bool b;
|- };
|+ } u;
| };
The testsuite still contains some examples of artificial restrictions
(see flat-union-clash-type.json, for example) that are no longer
technically necessary, now that there is no longer a collision between
enum tag values and non-variant member names; but fixing this will be
done in later patches, in part because some further changes are required
to keep QAPISchema*.check() from asserting. Also, a later patch will
add a reservation for the member name 'u' to avoid a collision between a
user's non-variant names and our internal choice of C union name.
Note, however, that we do not rename the generated enum, which
is still 'FooKind'. A further patch could generate implicit
enums as 'FooType', but while the generator already reserved
the '*Kind' namespace (commit 4dc2e69), there are already QMP
constructs with '*Type' naming, which means changing our
reservation namespace would have lots of churn to C code to
deal with a forced name change.
Backports commit e4ba22b31943ab02373359555bd7bcd66442632f from qemu
We have two issues with our qapi union layout:
1) Even though the QMP wire format spells the tag 'type', the
C code spells it 'kind', requiring some hacks in the generator.
2) The C struct uses an anonymous union, which places all tag
values in the same namespace as all non-variant members. This
leads to spurious collisions if a tag value matches a non-variant
member's name.
This patch is the front end for a series that converts to a
saner qapi union layout. By the end of the series, we will no
longer have the type/kind mismatch, and all tag values will be
under a named union, which requires clients to access
'obj->u.value' instead of 'obj->value'. But since the
conversion touches a number of files, it is easiest if we
temporarily support BOTH layouts simultaneously.
Given a simple union qapi type:
{ 'union':'Foo', 'data': { 'a':'int', 'b':'bool' } }
make the following changes in generated qapi-types.h:
| struct Foo {
|- FooKind kind;
|- union { /* union tag is @kind */
|+ union {
|+ FooKind kind;
|+ FooKind type;
|+ };
|+ union { /* union tag is @type */
| void *data;
| int64_t a;
| bool b;
|+ union { /* union tag is @type */
|+ void *data;
|+ int64_t a;
|+ bool b;
|+ } u;
| };
| };
Flat unions do not need the anonymous union for the tag member,
as we already fixed that to use the member name instead of 'kind'
back in commit 0f61af3e.
One additional change is needed in qapi.py: check_union() now
needs to check for collisions with 'type' in addition to those
with 'kind'.
Later, when the conversions are complete, we will remove the
duplication hacks, and also drop the check_union() restrictions.
Note, however, that we do not rename the generated enum, which
is still 'FooKind'. A further patch could generate implicit
enums as 'FooType', but while the generator already reserved
the '*Kind' namespace (commit 4dc2e69), there are already QMP
constructs with '*Type' naming, which means changing our
reservation namespace would have lots of churn to C code to
deal with a forced name change.
Backports commit f51d8fab44b231aa299d8de24cfdf9ba41ef4a21 from qemu
Rather than storing a base class as a pointer to a box, just
store the fields of that base class in the same order, so that
a child struct can be directly cast to its parent. This gives
less malloc overhead, less pointer dereferencing, and even less
generated code. Compare to the earlier commit 1e6c1616a "qapi:
Generate a nicer struct for flat unions" (although that patch
had fewer places to change, as less of qemu was directly using
qapi structs for flat unions). It also allows us to turn on
automatic type-safe wrappers for upcasting to the base class
of a struct.
Changes to the generated code look like this in qapi-types.h:
| struct SpiceChannel {
|- SpiceBasicInfo *base;
|+ /* Members inherited from SpiceBasicInfo: */
|+ char *host;
|+ char *port;
|+ NetworkAddressFamily family;
|+ /* Own members: */
| int64_t connection_id;
as well as additional upcast functions like qapi_SpiceChannel_base().
Meanwhile, changes to qapi-visit.c look like:
| static void visit_type_SpiceChannel_fields(Visitor *v, SpiceChannel **obj, Error **errp)
| {
| Error *err = NULL;
|
|- visit_type_implicit_SpiceBasicInfo(v, &(*obj)->base, &err);
|+ visit_type_SpiceBasicInfo_fields(v, (SpiceBasicInfo **)obj, &err);
| if (err) {
(the cast is necessary, since our upcast wrappers only deal with a
single pointer, not pointer-to-pointer); plus the wholesale
elimination of some now-unused visit_type_implicit_FOO() functions.
Without boxing, the corner case of one empty struct having
another empty struct as its base type now requires inserting a
dummy member (previously, the 'Base *base' member sufficed).
And now that we no longer consume a 'base' member in the generated
C struct, we can delete the former negative struct-base-clash-base
test.
Backports commit ddf21908961073199f3d186204da4810f2ea150b from qemu
A previous patch (commit 1e6c1616) made it possible to
directly cast from a qapi flat union type to its base type.
However, it requires the use of a C cast, which turns off
compiler type-safety checks. Fortunately, no such casts
exist, just yet.
Regardless, add inline type-safe wrappers named
qapi_FOO_base() for any union type FOO that has a base,
which can be used for a safer upcast, and enhance the
testsuite to cover the new functionality.
A future patch will extend the upcast support to structs,
where such conversions do exist already.
Note that C makes const-correct upcasts annoying because
it lacks overloads; these functions cast away const so that
they can accept user pointers whether const or not, and the
result in turn can be assigned to normal or const pointers.
Alternatively, this could have been done with macros, but
type-safe macros are hairy, and not worthwhile here.
This patch just adds upcasts. None of our code needed to
downcast from a base qapi class to a child. Also, in the
case of grandchildren (such as BlockdevOptionsQcow2), the
caller will need to call two functions to get to the inner
base (although it wouldn't be too hard to generate a
qapi_FOO_base_base() if desired). If a user changes qapi
to alter the base class hierarchy, such as going from
'A -> C' to 'A -> B -> C', it will change the type of
'qapi_C_base()', and the compiler will point out the places
that are affected by the new base.
One alternative was proposed, but was deemed too ugly to use
in practice: the generators could output redundant
information using anonymous types:
| struct Child {
| union {
| struct {
| Type1 parent_member1;
| Type2 parent_member2;
| };
| Parent base;
| };
| };
With that ugly proposal, for a given qapi type, obj->member
and obj->base.member would refer to the same storage; allowing
convenience in working with members without needing 'base.'
allowing typesafe upcast without needing a C cast by accessing
'&obj->base', and allowing downcasts from the parent back to
the child possible through container_of(obj, Child, base).
Backports commit 30594fe1cd4355626e73b80645428105d0df3cf6 from qemu
Move code from gen_union() into gen_struct_fields() in order for
a later patch to share code when enumerating inherited fields
for struct types.
No change to generated code.
Backports commit f87ab7f9bd956250c48b5c6e9b607b537fd21543 from qemu
A future patch will enable error reporting from the various
QAPISchema*.check() methods. But to report an error related
to an implicit type, we'll need to associate a location with
the type (the same location as the top-level entity that is
causing the creation of the implicit type), and once we do
that, keying off of whether foo.info exists is no longer a
viable way to determine if foo is an implicit type.
Instead, add an is_implicit() method to QAPISchemaEntity, and use it.
It can be overridden later for ObjectType and EnumType, when implicit
instances of those classes gain info.
Backports commit 49823c4b4304a3e4aa5d67e089946b12d6a52d64 from qemu
Previously, qapi-types and qapi-visit filtered out implicit
objects during visit_object_type() by using 'info' (works since
implicit objects do not [yet] have associated info); meanwhile
qapi-introspect filtered out all schema types on the first pass
by returning a python type from visit_begin(), which was then
used at a distance in QAPISchema.visit() to do the filtering.
Rather than keeping these ad hoc approaches, add a new visitor
callback visit_needed() which returns False to skip a given
entity, and which defaults to True unless overridden. Use the
new mechanism to simplify all three filtering visitors.
No change to the generated code.
Backports commit 25a0d9c977c2f5db914b0a1619759fd77d97b016 from qemu
Since we have consolidated all generated code to use 'err' as
the name of the local variable for error detection, we can
simplify the decision on whether to skip error detection (useful
for deallocation paths) to be a boolean.
Backports commit 18bdbc3ac8b477e160d56aa6ecd6942495ce44d0 from qemu
We had some pointless differences in the generated code for visit,
command marshalling, and events; unifying them makes it easier for
future patches to consolidate to common helper functions.
This is one patch of a series to clean up these differences.
This patch names the local visitor variable 'v' rather than 'm'.
Related objects, such as 'QapiDeallocVisitor', are also named by
their initials instead of an unrelated leading m.
No change in semantics to the generated code.
Backports commit f8b7f1a8eafa9f565ebecfe409e8741d38cd786b from qemu
It's first class, because unlike '**', it actually works, i.e. doesn't
require 'gen': false.
'**' will go away next.
Backports commit 28770e057f265a4e70bcbdfc2447cce7b5f2dc19 from qemu
Generate just 'FOO' instead of 'struct FOO' when possible.
Drop helper functions that are now unused.
Make pep8 and pylint reasonably happy.
Rename generate_FOO() functions to gen_FOO() for consistency.
Use more consistent and sensible variable names.
Consistently use c_ for mapping keys when their value is a C
identifier or type.
Simplify gen_enum() and gen_visit_union()
Consistently use single quotes for C text string literals.
Backports commit e98859a9b96d71dea8f9af43325edd43c7effe66 from qemu
Duplicated in commit 21cd70d. Yes, we can't import qapi-types, but
that's no excuse. Move the helpers from qapi-types.py to qapi.py, and
replace the duplicates in qapi-event.py.
The generated event enumeration type's lookup table becomes
const-correct (see commit 2e4450f), and uses explicit indexes instead
of relying on order (see commit 912ae9c).
Backports commit efd2eaa6c2992c214a13f102b6ddd4dca4697fb3 from qemu
Fixes flat unions to get the base's base members. Test case is from
commit 2fc0043, in qapi-schema-test.json:
{ 'union': 'UserDefFlatUnion',
'base': 'UserDefUnionBase',
'discriminator': 'enum1',
'data': { 'value1' : 'UserDefA',
'value2' : 'UserDefB',
'value3' : 'UserDefB' } }
{ 'struct': 'UserDefUnionBase',
'base': 'UserDefZero',
'data': { 'string': 'str', 'enum1': 'EnumOne' } }
{ 'struct': 'UserDefZero',
'data': { 'integer': 'int' } }
Patch's effect on UserDefFlatUnion:
struct UserDefFlatUnion {
/* Members inherited from UserDefUnionBase: */
+ int64_t integer;
char *string;
EnumOne enum1;
/* Own members: */
union { /* union tag is @enum1 */
void *data;
UserDefA *value1;
UserDefB *value2;
UserDefB *value3;
};
};
Flat union visitors remain broken. They'll be fixed next.
Code is generated in a different order now, but that doesn't matter.
The two guards QAPI_TYPES_BUILTIN_STRUCT_DECL and
QAPI_TYPES_BUILTIN_CLEANUP_DECL are replaced by just
QAPI_TYPES_BUILTIN.
Two ugly special cases for simple unions now stand out like sore
thumbs:
1. The type tag is named 'type' everywhere, except in generated C,
where it's 'kind'.
2. QAPISchema lowers simple unions to semantically equivalent flat
unions. However, the C generated for a simple unions differs from
the C generated for its equivalent flat union, and we therefore
need special code to preserve that pointless difference for now.
Mark both TODO.
Backports commit 2b162ccbe875e5323fc04c1009addbdea4d35220 from qemu
The QAPI code generators work with a syntax tree (nested dictionaries)
plus a few symbol tables (also dictionaries) on the side.
They have clearly outgrown these simple data structures. There's lots
of rummaging around in dictionaries, and information is recomputed on
the fly. For the work I'm going to do, I want more clearly defined
and more convenient interfaces.
Going forward, I also want less coupling between the back-ends and the
syntax tree, to make messing with the syntax easier.
Create a bunch of classes to represent QAPI schemata.
Have the QAPISchema initializer call the parser, then walk the syntax
tree to create the new internal representation, and finally perform
semantic analysis.
Shortcut: the semantic analysis still relies on existing check_exprs()
to do the actual semantic checking. All this code needs to move into
the classes. Mark as TODO.
Simple unions are lowered to flat unions. Flat unions and structs are
represented as a more general object type.
Catching name collisions in generated code would be nice. Mark as
TODO.
We generate array types eagerly, even though most of them aren't used.
Mark as TODO.
Nothing uses the new intermediate representation just yet, thus no
change to generated files.
Backports commit ac88219a6c78302c693fb60fe6cf04358540fbce from qemu
The camel_to_upper() method applies some heuristics to turn
a mixed case type name into an all-uppercase name. This is
used for example, to generate enum constant name prefixes.
The heuristics don't also generate a satisfactory name
though. eg
{ 'enum': 'QCryptoTLSCredsEndpoint',
'data': ['client', 'server']}
Results in Q_CRYPTOTLS_CREDS_ENDPOINT_CLIENT. This has
an undesirable _ after the initial Q and is missing an
_ between the CRYPTO & TLS strings.
Rather than try to add more and more heuristics to try
to cope with this, simply allow the QAPI schema to
specify the desired enum constant prefix explicitly.
eg
{ 'enum': 'QCryptoTLSCredsEndpoint',
'prefix': 'QCRYPTO_TLS_CREDS_ENDPOINT',
'data': ['client', 'server']}
Now gives the QCRYPTO_TLS_CREDS_ENDPOINT_CLIENT name.
Backports commit 351d36e454cddc67a1675740916636a7ccbf1c4b from qemu
Clean up white-space, brace placement, and superfluous #ifdef
QAPI_TYPES_BUILTIN_CLEANUP_DEF.
Backports commit 3a864e7c52af15017d5082a9ee39a7919f46d2b5 from qemu
Add a FIXME to remind us to fully audit whether removing the
'void *data' branch of each qapi union type can be done safely.
Backports commit ca56a822dd538017715345cbbe1f8829e0cc2742 from qemu
The struct generated for a flat union is weird: the members of its
base are at the end, except for the union tag, which is at the
beginning.
Example: qapi-schema-test.json has
{ 'struct': 'UserDefUnionBase',
'data': { 'string': 'str', 'enum1': 'EnumOne' } }
{ 'union': 'UserDefFlatUnion',
'base': 'UserDefUnionBase',
'discriminator': 'enum1',
'data': { 'value1' : 'UserDefA',
'value2' : 'UserDefB',
'value3' : 'UserDefB' } }
We generate:
struct UserDefFlatUnion
{
EnumOne enum1;
union {
void *data;
UserDefA *value1;
UserDefB *value2;
UserDefB *value3;
};
char *string;
};
Change to put all base members at the beginning, unadulterated. Not
only is this easier to understand, it also permits casting the flat
union to its base, if that should become useful.
We now generate:
struct UserDefFlatUnion
{
/* Members inherited from UserDefUnionBase: */
char *string;
EnumOne enum1;
/* Own members: */
union { /* union tag is @enum1 */
void *data;
UserDefA *value1;
UserDefB *value2;
UserDefB *value3;
};
};
Backports commit 1e6c1616a91cdcbe9a8387541f7689b8c11632aa from qemu
A flat union's tag member gets renamed to 'kind' in the generated
code. Breaks when another member named 'kind' exists.
Example, adapted from qapi-schema-test.json:
{ 'struct': 'UserDefUnionBase',
'data': { 'kind': 'str', 'enum1': 'EnumOne' } }
We generate:
struct UserDefFlatUnion
{
EnumOne kind;
union {
void *data;
UserDefA *value1;
UserDefB *value2;
UserDefB *value3;
};
char *kind;
};
Kill the silly rename.
Backports commit 0f61af3eb396ae163cd1572ce12e05f5d08d7c15 from qemu
gen_sync_call()'s parameter indent is useless: gen_sync_call() uses it
only as optional argument for push_indent() and pop_indent(), their
default is four, and gen_sync_call()'s only caller passes four. Drop
the parameter.
gen_visitor_input_containers_decl()'s parameter obj is always
"QOBJECT(args)". Use that, and drop the parameter.
Drop unused parameters of gen_marshal_output(),
gen_marshal_input_decl(), generate_visit_struct_body(),
generate_visit_list(), generate_visit_enum(), generate_declaration(),
generate_enum_declaration(), generate_decl_enum().
Drop unused variables in generate_event_enum_lookup(),
generate_enum_lookup(), generate_visit_struct_fields(), check_event().
Backports commit 5aa05d3f72e556752167f7005d6a3dea0f4432c5 from qemu
The enum string table parameters in various QOM/QAPI methods
are declared 'const char *strings[]'. This results in const
warnings if passed a variable that was declared as
static const char * const strings[] = { .... };
Add the extra const annotation to the parameters, since
neither the string elements, nor the array itself should
ever be modified.
Backports commit 2e4450ff432daef524cb3557fca68a3b7b5c7823 from qemu
Make sure that all generated C structs have at least one field; this
avoids potential issues with attempting to malloc space for
zero-length structs in C (g_malloc(sizeof struct) would return NULL).
It also avoids an incompatibility with C++ (where an empty struct is
size 1); that isn't important to us now but might be in future.
Generated empty structures look like this:
struct Abort
{
char qapi_dummy_field_for_empty_struct;
};
This silences clang warnings like:
./qapi-types.h:3752:1: warning: empty struct has size 0 in C, size 1 in C++ [-Wextern-c-compat]
struct Abort
^
Backports commit 83ecb22ba2c91a4674ae109595a8ed1da8de4d7a from qemu
Useless, because it can only occur in commands, and we're not dealing
with commands here.
Backports commit 4f3568002393380558705397bda4cd5f224ffe29 from qemu
Enhance the testsuite to cover downstream alternates, including
whether the branch name or type is downstream. Update the
generator to mangle alternate names in the appropriate places.
Backports commit d1f07c86c05706facf950b0b0dba370f71fd5ef6 from qemu
Enhance the testsuite to cover downstream flat unions, including
the base type, discriminator name and type, and branch name and
type. Update the generator to mangle the union names in the
appropriate places.
Backports commit 857af5f06c3fb097d1bb6bc8a23b9992aac99e75 from qemu
