This patch introduces a helper to query the iotlb entry for a
possible iova. This will be used by later device IOTLB API to enable
the capability for a dataplane (e.g vhost) to query the IOTLB.
Backports commit 052c8fa9983f553fdfa0d61034774070dd639c2b from qemu
Device models often have to perform multiple access to a single
memory region that is known in advance, but would to use "DMA-style"
functions instead of address_space_map/unmap. This can happen
for example when the data has to undergo endianness conversion.
Introduce a new data structure to cache the result of
address_space_translate without forcing usage of a host address
like address_space_map does.
Backports commit 1f4e496e1fc2eb6c8bf377a0f9695930c380bfd3 from qemu
This extracts the common part of address_space_map and
address_space_cache_init into a new function.
Backports commit 715c31ec8e12107f47ac74b464c97e813c76f898 from qemu
Templatize the address_space_* and *_phys functions, so that we can add
similar functions in the next patch that work with a lightweight,
cache-like version of address_space_map/unmap.
Backports commit 0ce265ffef87f19f4dd1ff0663e09a63d66ae408 from qemu
rcu_read_unlock was not called if the address_space_access_valid result is
negative.
This caused (at least) a problem when qemu on PPC/E500+TAP failed to terminate
properly and instead got stuck in a deadlock.
Backports commit 662a97d74f9b34cafe9aeb6d96620a97d768a1fa from qemu
A bug (1647683) was reported showing a crash when removing
breakpoints. The reproducer was bisected to 3359baad when tb_flush
was finally made thread safe. While in MTTCG the locking in
breakpoint_invalidate would have prevented any problems, but
currently tb_lock() is a NOP for system emulation.
The race is between a tb_flush from the gdbstub and the
tb_invalidate_phys_addr() in breakpoint_invalidate().
Ideally we'd have actual locking here; for the moment the
simple fix is to do a full tb_flush() for a bp invalidate,
since that is thread-safe even if no lock is taken.
Backports commit a9353fe897ca2687e5b3385ed39e3db3927a90e0 from qemu
The memory_dispatch field is meant to be protected by RCU so we should
use the correct primitives when accessing it. This race was flagged up
by the ThreadSanitizer.
Backports commit f35e44e7645edbb08e35b111c10c2fc57e2905c7 from qemu
In the linux-user case all things that involve ''l1_map' and PageDesc
tweaks are protected by the memory lock (mmpa_lock). For SoftMMU mode
we previously relied on single threaded behaviour, with MTTCG we now use
the tb_lock().
As a result we need to do a little re-factoring and push the taking of
this lock up the call tree. This requires a slightly different entry for
the SoftMMU and user-mode cases from tb_invalidate_phys_range.
This also means user-mode breakpoint insertion needs to take two locks
but it hadn't taken any previously so this is an improvement.
Backpoirts commit ba051fb5e56d5ff5e4fa672d37954452e58543b2 from qemu
Support target CPUs having a page size which isn't knownn
at compile time. To use this, the CPU implementation should:
* define TARGET_PAGE_BITS_VARY
* not define TARGET_PAGE_BITS
* define TARGET_PAGE_BITS_MIN to the smallest value it
might possibly want for TARGET_PAGE_BITS
* call set_preferred_target_page_bits() in its realize
function to indicate the actual preferred target page
size for the CPU (and report any error from it)
In CONFIG_USER_ONLY, the CPU implementation should continue
to define TARGET_PAGE_BITS appropriately for the guest
OS page size.
Machines which want to take advantage of having the page
size something larger than TARGET_PAGE_BITS_MIN must
set the MachineClass minimum_page_bits field to a value
which they guarantee will be no greater than the preferred
page size for any CPU they create.
Note that changing the target page size by setting
minimum_page_bits is a migration compatibility break
for that machine.
For debugging purposes, attempts to use TARGET_PAGE_SIZE
before it has been finally confirmed will assert.
Backports commit 20bccb82ff3ea09bcb7c4ee226d3160cab15f7da from qemu
Allocate sub_section dynamically. Remove dependency
on TARGET_PAGE_SIZE to make run-time page size detection
for arm platforms.
Backports commit 2615fabd42ea0078dd9e659bdb21a5b7a1f87a9a from qemu
this is the first step in reducing the brk heap fragmentation
created by the map->nodes memory allocation. Since the introduction
of RCU the freeing of the PhysPageMaps is delayed so that sometimes
several hundred are allocated at the same time.
Even worse the memory for map->nodes is allocated and shortly
afterwards reallocated. Since the number of nodes it grows
to in the end is the same for all PhysPageMaps remember this value
and at least avoid the reallocation.
The large number of simultaneous allocations (about 450 x 70kB in
my configuration) has to be addressed later.
Backports commit 101420b886eec36990419bc9ed5b503622af8a0d from qemu
Tracked down with an ugly, brittle and probably buggy Perl script.
Also move includes converted to <...> up so they get included before
ours where that's obviously okay.
Backports commit a9c94277f07d19d3eb14f199c3e93491aa3eae0e from qemu
Let users of qemu_get_ram_ptr and qemu_ram_ptr_length pass in an
address that is relative to the MemoryRegion. This basically means
what address_space_translate returns.
Because the semantics of the second parameter change, rename the
function to qemu_map_ram_ptr.
Backports commit 0878d0e11ba8013dd759c6921cbf05ba6a41bd71 from qemu
Move the old qemu_ram_addr_from_host to memory_region_from_host and
make it return an offset within the region. For qemu_ram_addr_from_host
return the ram_addr_t directly, similar to what it was before
commit 1b5ec23 ("memory: return MemoryRegion from qemu_ram_addr_from_host",
2013-07-04).
Backports commit 07bdaa4196b51bc7ffa7c3f74e9e4a9dc8a7966a from qemu
Of the two callers, one does not use it, and the other can compute
it itself based on the other output argument (offset) and the RAMBlock.
Backports commit f615f39616c4fd1a3a3b078af8d75bb4be6390de from qemu
Remove direct uses of ram_addr_t and optimize memory_region_{get,set}_fd
now that a MemoryRegion knows its RAMBlock directly.
Backports commit 4ff87573df3606856a92c14eef3393a63d736d11 from qemu
mr->ram_block->offset is already aligned to both host and target size
(see qemu_ram_alloc_internal). Remove further masking as it is
unnecessary.
Backports commit e4e697940dff612b789b0858270c20a8b680f78d from qemu
On the one hand, we have already qemu_get_ram_block() whose function
is similar. On the other hand, we can directly use mr->ram_block but
searching RAMblock by ram_addr which is a kind of waste.
Backports commit fa53a0e53efdc7002497ea4a76aacf6cceb170ef from qemu
exec-all.h contains TCG-specific definitions. It is not needed outside
TCG-specific files such as translate.c, exec.c or *helper.c.
One generic function had snuck into include/exec/exec-all.h; move it to
include/qom/cpu.h.
Backports commit 63c915526d6a54a95919ebece83fa9ca631b2508 from qemu
Move the architecture agnostic function prototypes for exec.c out of
cputlb.h to exec-all.h. This allows hiding of the arch specific
cputlb.h from exec.c which should be getting close to having no
architecture specifics. Prepares support for multi-arch, which will have
a minimal cpu.h that services exec.c but not cputlb.h.
Backports commit dfccc7602374c9fd3b083208b552d62daa244811 from qemu
To prepare for multi-arch, cputlb.c should only have awareness of one
single architecture. This means it should not have access to the full
CPU lists which may be heterogeneous. Instead, push the CPU_LOOP() up
to the one and only caller in exec.c.
Backports commit 9a13565d52bfd321934fb44ee004bbaf5f5913a8 from qemu
Although accesses to ram_list.dirty_memory[] use atomics so multiple
threads can safely dirty the bitmap, the data structure is not fully
thread-safe yet.
This patch handles the RAM hotplug case where ram_list.dirty_memory[] is
grown. ram_list.dirty_memory[] is change from a regular bitmap to an
RCU array of pointers to fixed-size bitmap blocks. Threads can continue
accessing bitmap blocks while the array is being extended. See the
comments in the code for an in-depth explanation of struct
DirtyMemoryBlocks.
I have tested that live migration with virtio-blk dataplane works.
Backports commit 5b82b703b69acc67b78b98a5efc897a3912719eb from qemu
Move declarations out of qemu-common.h for functions declared in
utils/ files: e.g. include/qemu/path.h for utils/path.c.
Move inline functions out of qemu-common.h and into new files (e.g.
include/qemu/bcd.h)
Backports commit f348b6d1a53e5271cf1c9f9acc4646b4b98c1771 from qemu
Commit 57cb38b included qapi/error.h into qemu/osdep.h to get the
Error typedef. Since then, we've moved to include qemu/osdep.h
everywhere. Its file comment explains: "To avoid getting into
possible circular include dependencies, this file should not include
any other QEMU headers, with the exceptions of config-host.h,
compiler.h, os-posix.h and os-win32.h, all of which are doing a
similar job to this file and are under similar constraints."
qapi/error.h doesn't do a similar job, and it doesn't adhere to
similar constraints: it includes qapi-types.h. That's in excess of
100KiB of crap most .c files don't actually need.
Add the typedef to qemu/typedefs.h, and include that instead of
qapi/error.h. Include qapi/error.h in .c files that need it and don't
get it now. Include qapi-types.h in qom/object.h for uint16List.
Update scripts/clean-includes accordingly. Update it further to match
reality: replace config.h by config-target.h, add sysemu/os-posix.h,
sysemu/os-win32.h. Update the list of includes in the qemu/osdep.h
comment quoted above similarly.
This reduces the number of objects depending on qapi/error.h from "all
of them" to less than a third. Unfortunately, the number depending on
qapi-types.h shrinks only a little. More work is needed for that one.
Backports commit da34e65cb4025728566d6504a99916f6e7e1dd6a from qemu
After reporting an error, ram_block_add was going on with the registration
of the RAMBlock. The visible effect is that it unlocked the ramlist
mutex twice.
Backports commit 39c350ee12e733070e63d64a21bd42607366ea99 from qemu
Under heavy workloads the lookup will likely end up with the same
MemoryRegionSection from last time. Using a pointer to cache the result,
like ram_list.mru_block, significantly reduces cost of
address_space_translate.
During address space topology update, as->dispatch will be reallocated
so the pointer is invalidated automatically.
Perf reports a visible drop on the cpu usage, because phys_page_find is
not called. Before:
2.35% qemu-system-x86_64 [.] phys_page_find
0.97% qemu-system-x86_64 [.] address_space_translate_internal
0.95% qemu-system-x86_64 [.] address_space_translate
0.55% qemu-system-x86_64 [.] address_space_lookup_region
After:
0.97% qemu-system-x86_64 [.] address_space_translate_internal
0.97% qemu-system-x86_64 [.] address_space_lookup_region
0.84% qemu-system-x86_64 [.] address_space_translate
Backports commit 729633c2bc30496073431584eb6e304776b4ebd4 from qemu
This will be shared by the next patch.
Also add a comment explaining the unobvious condition on "size.hi".
Backports commit 29cb533d8cbff1330717619780c2f1dfe764e003 from qemu
All references to mr->ram_addr are replaced by
memory_region_get_ram_addr(mr) (except for a few assertions that are
replaced with mr->ram_block).
Backports commit 8e41fb63c5bf29ecabe0cee1239bf6230f19978a from qemu
We don't force "const" qualifiers with pointers in QEMU, but it's still
good to keep a clean function interface. Assigning to mr->ram_block is
in this sense ugly - one initializer mutating its owning object's state.
Move it to memory_region_init_*, where mr->ram_addr is assigned.
Backports commit 0a75601853c00f3729fa62c49ec0d4bb1e3d9bc1 from qemu
Previously we return RAMBlock.offset; now return the pointer to the
whole structure.
ram_block_add returns void now, error is completely passed with errp.
Backports commit 528f46af6ecd1e300db18684969104d4067b867b from qemu
these two functions consume too much cpu overhead to
find the RAMBlock by ram address.
After this patch, we can pass the RAMBlock pointer
to them so that they don't need to find the RAMBlock
anymore most of the time. We can get better performance
in address translation processing.
Backports commit 3655cb9c7375a595a8051ec677c515b24d5c1fe6 from qemu
Each RAM memory region has a unique corresponding RAMBlock.
In the current realization, the memory region only stored
the ram_addr which means the offset of RAM address space,
We need to qurey the global ram.list to find the ram block
by ram_addr if we want to get the ram block, which is very
expensive.
Now, we store the RAMBlock pointer into memory region
structure. So, if we know the mr, we can easily get the
RAMBlock.
Backports commit 58eaa2174e99d9a05172d03fd2799ab8fd9e6f60 from qemu
Clean up includes so that osdep.h is included first and headers
which it implies are not included manually.
This commit was created with scripts/clean-includes.
Backports commit 7b31bbc2e68605ab2f10dc609dd54cf4c7b5f49a from qemu
address_space_translate_internal will clamp the *plen length argument
based on the size of the memory region being queried. The iommu walker
logic in addresss_space_translate was ignoring this by discarding the
post fn call value of *plen. Fix by just always using *plen as the
length argument throughout the fn, removing the len local variable.
This fixes a bootloader bug when a single elf section spans multiple
QEMU memory regions.
Backports commit 23820dbfc79d1c9dce090b4c555994f2bb6a69b3 from qemu
Add a MemoryRegion property, which if set is used to construct
the CPU's initial (default) AddressSpace.
Backports commit 6731d864f80938e404dc3e5eb7f6b76b891e3e43 from qemu
When accessing the dispatch pointer in an AddressSpace within an RCU
critical section we should always use atomic_rcu_read(). Fix an
access within memory_region_section_get_iotlb() which was incorrectly
doing a direct pointer access.
Backports commit 0b8e2c1002afddc8ef3d52fa6fc29e4768429f98 from qemu
check the return value of the function it calls and error if it's non-0
Fixup qemu_rdma_init_one_block that is the only current caller,
and rdma_add_block the only function it calls using it.
Pass the name of the ramblock to the function; helps in debugging.
Backports commit e3807054e20fb3b94d18cb751c437ee2f43b6fac from qemu
It is not necessary to munmap an area before remapping it with MAP_FIXED;
if the memory region specified by addr and len overlaps pages of any
existing mapping, then the overlapped part of the existing mapping will
be discarded.
On the other hand, if QEMU does munmap the pages, there is a small
probability that another mmap sneaks in and catches the just-freed
portion of the address space. In effect, munmap followed by
mmap(MAP_FIXED) is a use-after-free error, and Coverity flags it
as such. Fix it.
Backports commit f18c69cfc554cf9776eb3c35b7510e17541afacb from qemu
This reverts commit c3c1bb9.
It causes problems with boards that declare memory regions shorter
than the registers they contain.
Backports commit 4025446f0ac6213335c22ec43f3c3d8362ce7286 from qemu
Block size must fundamentally be a multiple of target page size.
Aligning automatically removes need to worry about the alignment
from callers.
Note: the only caller of qemu_ram_resize (acpi) already happens to have
size padded to a power of 2, but we would like to drop the padding in
ACPI core, and don't want to expose target page size knowledge to ACPI
Backports commit 129ddaf31be583fb7c97812e07e028661005ce42 from qemu
Allow "unlocked" reads of the ram_list by using an RCU-enabled QLIST.
The ramlist mutex is kept. call_rcu callbacks are run with the iothread
lock taken, but that may change in the future. Writers still take the
ramlist mutex, but they no longer need to assume that the iothread lock
is taken.
Readers of the list, instead, no longer require either the iothread
or ramlist mutex, but they need to use rcu_read_lock() and
rcu_read_unlock().
One place in arch_init.c was downgrading from write side to read side
like this:
qemu_mutex_lock_iothread()
qemu_mutex_lock_ramlist()
...
qemu_mutex_unlock_iothread()
...
qemu_mutex_unlock_ramlist()
and the equivalent idiom is:
qemu_mutex_lock_ramlist()
rcu_read_lock()
...
qemu_mutex_unlock_ramlist()
...
rcu_read_unlock()
Backports the write barriers from commit 0dc3f44aca18b1be8b425f3f4feb4b3e8d68de2e in qemu
Coverity flags this as "dereference after null check". Not quite a
dereference, since it will just EFAULT, but still nice to fix.
Backports commit a904c91196a9c5dbd7b9abcd3d40b0824286fb1c from qemu
The TARGET_HAS_ICE #define is intended to indicate whether a target-*
guest CPU implementation supports the breakpoint handling. However,
all our guest CPUs have that support (the only two which do not
define TARGET_HAS_ICE are unicore32 and openrisc, and in both those
cases the bp support is present and the lack of the #define is just
a bug). So remove the #define entirely: all new guest CPU support
should include breakpoint handling as part of the basic implementation.
Backports commit ec53b45bcd1f74f7a4c31331fa6d50b402cd6d26 from qemu
