The next commit wants to generate qapi-emit-events.{c.h}. To enable
that, extend QAPISchemaModularCVisitor to support additional "system
modules", i.e. modules that don't correspond to a (user-defined) QAPI
schema module.
Backports commit c2e196a9b41235a308fb6d1c516aa91ba0a807c8 from qemu
We neglect to call .visit_module() for the special module we use for
built-ins. Harmless, but clean it up anyway. The
tests/qapi-schema/*.out now show the built-in module as 'module None'.
Subclasses of QAPISchemaModularCVisitor need to ._add_module() this
special module to enable code generation for built-ins. When this
hasn't been done, QAPISchemaModularCVisitor.visit_module() does
nothing for the special module. That looks like built-ins could
accidentally be generated into the wrong module when a subclass
neglects to call ._add_module(). Can't happen, because built-ins are
all visited before any other module. But that's non-obvious. Switch
off code generation explicitly.
Rename QAPISchemaModularCVisitor._begin_module() to
._begin_user_module().
New QAPISchemaModularCVisitor._is_builtin_module(), for clarity.
Backports commit dcac64711ea906e844ae60a5927e5580f7252c1e from qemu
Default branches variant should use the member conditional.
This fixes compilation with --disable-replication.
Fixes: 335d10cd8e2c3bb6067804b095aaf6371fc1983e
Backports commit ce1a1aec47877a281d69dbc2e65f86bfe8fea231 from qemu
The generated C enumeration types explicitly set the enumeration
constants to 0, 1, 2, ... That's exactly what you get when you don't
supply values.
Drop the explicit values. No change now, but it will avoid gaps in
the values when we later add support for 'if' conditions. Avoiding
such gaps will save us the trouble of changing the ENUM_lookup[]
tables to work without a sentinel.
We'll have to take care to ensure the headers required by the 'if'
conditions get always included before the generated QAPI code.
Fortunately, our convention to include "qemu/osdep.h" first in any .c
ensures that's the case for our CONFIG_FOO macros
Backports commit 9c2f56e9f9d5a1f9ddac77dda35f997738e85d11 from qemu
Rename QAPISchemaEnumType.values and related variables to members.
Makes sense ever since commit 93bda4dd4 changed .values from list of
string to list of QAPISchemaMember. Obvious no-op.
Backports commit 57516863644817ca59fab023e0c68d139929f3e0 from qemu
We emit a dummy variable in each .c file "to shut up OSX toolchain
warnings about empty .o files" (commit 252dc3105fc). Separate it from
the code preceding it (if any) with a blank line.
Backports commit 5f1450f5444b42cde4b1edc61ea5fdcd57404d3e from qemu
The generated qapi_event_send_FOO() take an Error ** argument. They
can't actually fail, because all they do with the argument is passing it
to functions that can't fail: the QObject output visitor, and the
@qmp_emit callback, which is either monitor_qapi_event_queue() or
event_test_emit().
Drop the argument, and pass &error_abort to the QObject output visitor
and @qmp_emit instead.
Backports commit 3ab72385b21d8d66df3f5fea42097ce264dc9d6b from qemu
build_params() returns '' instead of 'void' when there are no
parameters. Can't happen now, but the next commit will change that.
Backports commit bdd2d42b890b3a908fa3fbdc9661541e1b57eb15 from qemu
Fix the following issues:
common.py:873:13: E129 visually indented line with same indent as next logical line
common.py:1766:5: E741 ambiguous variable name 'l'
common.py:1784:1: E305 expected 2 blank lines after class or function definition, found 1
common.py:1833:1: E305 expected 2 blank lines after class or function definition, found 1
common.py:1843:1: E305 expected 2 blank lines after class or function definition, found 1
visit.py:181:18: E127 continuation line over-indented for visual indent
Backports commit b736e25a1820c63f7d69baa03e624cef80c4de90 from qemu
Types & visitors are coupled and must be handled together to avoid
temporary build regression.
Wrap generated types/visitor code with #if/#endif using the context
helpers. Derived from a patch by Marc-André.
Backports commit 9f88c66211342714b06c051140fd64ffd338dbe1 from qemu
Wrap generated code with #if/#endif using an 'ifcontext' on
QAPIGenCSnippet objects.
This makes a conditional event's qapi_event_send_FOO() compile-time
conditional, but its enum QAPIEvent member remains unconditional for
now. A follow up patch "qapi-event: add 'if' condition to implicit
event enum" will improve this.
Backports commit c3cd6aa0201c126eda8dc71b60e7aa259a3e79b9 from qemu
Add helpers to wrap generated code with #if/#endif lines.
A later patch wants to use QAPIGen for generating C snippets rather
than full C files with copyright headers etc. Splice in class
QAPIGenCCode between QAPIGen and QAPIGenC.
Add a 'with' statement context manager that will be used to wrap
generator visitor methods. The manager will check if code was
generated before adding #if/#endif lines on QAPIGenCSnippet
objects. Used in the following patches.
Backports commit ded9fc28b5a07213f3e5e8ac7ea0494b85813de1 from qemu
Skip preprocessor lines when adding indentation, since that would
likely result in invalid code.
Backports commit 485d948ce86f5a096dc848ec31b70cd66452d40d from qemu
We commonly initialize attributes to None in .init(), then set their
real value in .check(). Accessing the attribute before .check()
yields None. If we're lucky, the code that accesses the attribute
prematurely chokes on None.
It won't for .ifcond, because None is a legitimate value.
Leave the ifcond attribute undefined until check().
Backports commit 4fca21c1b043cb1ef2e197ef15e7474ba668d925 from qemu
Built-in objects remain unconditional. Explicitly defined objects use
the condition specified in the schema. Implicitly defined objects
inherit their condition from their users. For most of them, there is
exactly one user, so the condition to use is obvious. The exception
is wrapped types generated for simple union variants, which can be
shared by any number of simple unions. The tight condition would be
the disjunction of the conditions of these simple unions. For now,
use the wrapped type's condition instead. Much simpler and good
enough for now.
Backports commit 2cbc94376e718448699036be7f6e29ab75312b70 from qemu
Accept 'if' key in top-level elements, accepted as string or list of
string type. The following patches will modify the test visitor to
check the value is correctly saved, and generate #if/#endif code (as a
single #if/endif line or a series for a list).
Example of 'if' key:
{ 'struct': 'TestIfStruct', 'data': { 'foo': 'int' },
'if': 'defined(TEST_IF_STRUCT)' }
The generated code is for now *unconditional*. Later patches generate
the conditionals.
Backports commit 967c885108f18e5065744719f7959ba5ea0a5b0d from qemu
New option will be used to allow commands, which are prepared/need
to run, during preconfig state. Other commands that should be able
to run in preconfig state, should be amended to not expect machine
in initialized state or deal with it.
For compatibility reasons, commands that don't use new flag
'allow-preconfig' explicitly are not permitted to run in
preconfig state but allowed in all other states like they used
to be.
Within this patch allow following commands in preconfig state:
qmp_capabilities
query-qmp-schema
query-commands
query-command-line-options
query-status
exit-preconfig
to allow qmp connection, basic introspection and moving to the next
state.
PS:
set-numa-node and query-hotpluggable-cpus will be enabled later in
a separate patches.
Backports commit d6fe3d02e9a2ce7b63a0723d0b71f3013f59d705 from qemu
It was missed in the first version of OOB series. We should check this
to make sure we throw the right error when fault value is passed in.
Backports commit 9408860165e07aaadec66c336f3dc849b945a8ed from qemu
Here "oob" stands for "Out-Of-Band". When "allow-oob" is set, it means
the command allows out-of-band execution.
The "oob" idea is proposed by Markus Armbruster in following thread:
https://lists.gnu.org/archive/html/qemu-devel/2017-09/msg02057.html
This new "allow-oob" boolean will be exposed by "query-qmp-schema" as
well for command entries, so that QMP clients can know which commands
can be used in out-of-band calls. For example the command "migrate"
originally looks like:
{"name": "migrate", "ret-type": "17", "meta-type": "command",
"arg-type": "86"}
And it'll be changed into:
{"name": "migrate", "ret-type": "17", "allow-oob": false,
"meta-type": "command", "arg-type": "86"}
This patch only provides the QMP interface level changes. It does not
contain the real out-of-band execution implementation yet.
Backports commit 876c67512e2af8c05686faa9f9ff49b38d7a392c from qemu
Python 2 happily reads UTF-8 files in text mode, but Python 3 requires
either UTF-8 locale or an explicit encoding passed to open(). Commit
d4e5ec877ca fixed this by setting the en_US.UTF-8 locale. Falls apart
when the locale isn't be available.
Matthias Maier and Arfrever Frehtes Taifersar Arahesis proposed to use
binary mode instead, with manual conversion from bytes to str. Works,
but opening with an explicit encoding is simpler, so do that.
Since Python 2's open() doesn't support the encoding parameter, we
need to suppress it with a version check.
Backports commit de685ae5e9a4b523513033bd6cadc8187a227170 from qemu
It often happens that just a few discriminator values imply extra data in
a flat union. Existing checks did not make possible to leave other values
uncovered. Such cases had to be worked around by either stating a dummy
(empty) type or introducing another (subset) discriminator enumeration.
Both options create redundant entities in qapi files for little profit.
With this patch it is not necessary anymore to add designated union
fields for every possible value of a discriminator enumeration.
Backports commit 800877bb1639d38ffaebe312a37b61c66bb10c83 from qemu
The event generator produces an enum, and put it in the last visited
module. It fits better in the main module, since it's the set of all
visited events, from all modules.
Backports commit f030ffd39d6c1ea8fff281be5e4b19c819d7ce10 from qemu
Now that we can safely call QOBJECT() on QObject * as well as its
subtypes, we can have macros qobject_ref() / qobject_unref() that work
everywhere instead of having to use QINCREF() / QDECREF() for QObject
and qobject_incref() / qobject_decref() for its subtypes.
The replacement is mechanical, except I broke a long line, and added a
cast in monitor_qmp_cleanup_req_queue_locked(). Unlike
qobject_decref(), qobject_unref() doesn't accept void *.
Note that the new macros evaluate their argument exactly once, thus no
need to shout them.
Backports commit cb3e7f08aeaab0ab13e629ce8496dca150a449ba from qemu
Move qapi-schema.json to qapi/, so it's next to its modules, and all
files get generated to qapi/, not just the ones generated for modules.
Consistently name the generated files qapi-MODULE.EXT:
qmp-commands.[ch] become qapi-commands.[ch], qapi-event.[ch] become
qapi-events.[ch], and qmp-introspect.[ch] become qapi-introspect.[ch].
This gets rid of the temporary hacks in scripts/qapi/commands.py,
scripts/qapi/events.py, and scripts/qapi/common.py.
Backports commit eb815e248f50cde9ab86eddd57eca5019b71ca78 from qemu
In my "build everything" tree, a change to the types in
qapi-schema.json triggers a recompile of about 4800 out of 5100
objects.
The previous commit split up qmp-commands.h, qmp-event.h, qmp-visit.h,
qapi-types.h. Each of these headers still includes all its shards.
Reduce compile time by including just the shards we actually need.
To illustrate the benefits: adding a type to qapi/migration.json now
recompiles some 2300 instead of 4800 objects. The next commit will
improve it further.
Backports commit 9af2398977a78d37bf184d6ff6bd04c72bfbf006 from qemu
Our qapi-schema.json is composed of modules connected by include
directives, but the generated code is monolithic all the same: one
qapi-types.h with all the types, one qapi-visit.h with all the
visitors, and so forth. These monolithic headers get included all
over the place. In my "build everything" tree, adding a QAPI type
recompiles about 4800 out of 5100 objects.
We wouldn't write such monolithic headers by hand. It stands to
reason that we shouldn't generate them, either.
Split up generated qapi-types.h to mirror the schema's modular
structure: one header per module. Name the main module's header
qapi-types.h, and sub-module D/B.json's header D/qapi-types-B.h.
Mirror the schema's includes in the headers, so that qapi-types.h gets
you everything exactly as before. If you need less, you can include
one or more of the sub-module headers. To be exploited shortly.
Split up qapi-types.c, qapi-visit.h, qapi-visit.c, qmp-commands.h,
qmp-commands.c, qapi-event.h, qapi-event.c the same way.
qmp-introspect.h, qmp-introspect.c and qapi.texi remain monolithic.
The split of qmp-commands.c duplicates static helper function
qmp_marshal_output_str() in qapi-commands-char.c and
qapi-commands-misc.c. This happens when commands returning the same
type occur in multiple modules. Not worth avoiding.
Since I'm going to rename qapi-event.[ch] to qapi-events.[ch], and
qmp-commands.[ch] to qapi-commands.[ch], name the shards that way
already, to reduce churn. This requires temporary hacks in
commands.py and events.py. Similarly, c_name() must temporarily
be taught to munge '/' in common.py. They'll go away with the rename.
Backports commit 252dc3105fc494182e236e97fe20f2d6b1d652cb from qemu
guardname() fails to return a valid C identifier for arguments
containing anything but [A-Za-z0-9_.-']. Fix that. Don't bother
protecting ticklish identifiers; header guards are all-caps, and no
ticklish identifiers are.
Backports commit f9c146399dabefb8cd13c9c467a9e710af15ea70 from qemu
Linking code from multiple separate QAPI schemata into the same
program is possible, but involves some weirdness around built-in
types:
* We generate code for built-in types into .c only with option
--builtins. The user is responsible for generating code for exactly
one QAPI schema per program with --builtins.
* We generate code for built-in types into .h regardless of
--builtins, but guarded by #ifndef QAPI_VISIT_BUILTIN. Because all
copies of this code are exactly the same, including any combination
of these headers works.
Replace this contraption by something more conventional: generate code
for built-in types into their very own files: qapi-builtin-types.c,
qapi-builtin-visit.c, qapi-builtin-types.h, qapi-builtin-visit.h, but
only with --builtins. Obey --output-dir, but ignore --prefix for
them.
Make qapi-types.h include qapi-builtin-types.h. With multiple
schemata you now have multiple qapi-types.[ch], but only one
qapi-builtin-types.[ch]. Same for qapi-visit.[ch] and
qapi-builtin-visit.[ch].
Bonus: if all you need is built-in stuff, you can include a much
smaller header. To be exploited shortly.
Backports commit cdb6610ae4283720037bae2af1f78bd40eb5fe71 from qemu
The use of QAPIGen is rather shallow so far: most of the output
accumulation is not converted. Take the next step: convert output
accumulation in the code-generating visitor classes. Helper functions
outside these classes are not converted.
Backports commit 71b3f0459c460c9e16a47372ccddbfa6e2c7aadf from qemu
The include directive permits modular QAPI schemata, but the generated
code is monolithic all the same. To permit generating modular code,
the front end needs to pass more information on inclusions to the back
ends. The commit before last added the necessary information to the
parse tree. This commit adds it to the intermediate representation
and its QAPISchemaVisitor. A later commit will use this to to
generate modular code.
New entity QAPISchemaInclude represents inclusions. Call new visitor
method visit_include() for it, so visitors can see the sub-modules a
module includes.
Note that unlike other entities, QAPISchemaInclude has no name, and is
therefore not added to entity_dict.
New QAPISchemaEntity attribute @module names the entity's source file.
Call new visitor method visit_module() when it changes during a visit,
so visitors can keep track of the module being visited.
Backports commit cf40a0a5c2e1091846974cc8cc95a60e0b1db4af from qemu
The generators' conversion to visitors (merge commit 9e72681d16)
changed the processing order of entities from source order to
alphabetical order. The next commit needs source order, so change it
back.
Backports commit 8a84767cc4f7e00e5dd62435c32be9e7d2cbe4d3 from qemu
The parse tree is a list of expressions. Except include expressions
currently get replaced by the included file's parse tree.
Instead of throwing away the include expression, keep it with the file
name expanded so you don't have to track the including file's
directory to make sense of it.
A future commit will put this include expression to use.
Backports commit 97f0249474d19c1d60fb9d934c8bc08625a619ca from qemu
Error messages print absolute file names of included files even if the
user gave a relative one on the command line:
$ PYTHONPATH=scripts python -B tests/qapi-schema/test-qapi.py tests/qapi-schema/include-cycle.json
In file included from tests/qapi-schema/include-cycle.json:1:
In file included from /work/armbru/qemu/tests/qapi-schema/include-cycle-b.json:1:
/work/armbru/qemu/tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json
Improve this to
In file included from tests/qapi-schema/include-cycle.json:1:
In file included from tests/qapi-schema/include-cycle-b.json:1:
tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json
The error message when an include file can't be opened prints the
include directive's file name, which is relative to the including
file. Change this to print the file name relative to the working
directory. Visible in tests/qapi-schema/include-no-file.err.
Backports commit af97502ce9c648ae5c746b9e562d6e4586f02eee from qemu
A massive number of objects depends on QAPI-generated headers. In my
"build everything" tree, it's roughly 4800 out of 5100. This is
particularly annoying when only some of the generated files change,
say for a doc fix.
Improve qapi-gen.py to touch its output files only if they actually
change. Rebuild time for a QAPI doc fix drops from many minutes to a
few seconds. Rebuilds get faster for certain code changes, too. For
instance, adding a simple QMP event now recompiles less than 200
instead of 4800 objects. But adding a QAPI type is as bad as ever;
we've clearly got more work to do.
Backports commit 907b846653fb3757bf2ab98d6d66f92df34d875f from qemu
Whenever qapi-schema.json changes, we run six programs eleven times to
update eleven files. Similar for qga/qapi-schema.json. This is
silly. Replace the six programs by a single program that spits out
all eleven files.
The programs become modules in new Python package qapi, along with the
helper library. This requires moving them to scripts/qapi/. While
moving them, consistently drop executable mode bits.
Backports commit fb0bc835e56b894cbc7236294921e5393c786ad8 from qemu
