Commit Graph

5147 Commits

Author SHA1 Message Date
Richard Henderson
3cb68bc44e target/arm: Move helper_dc_zva to helper-a64.c
This is an aarch64-only function. Move it out of the shared file.
This patch is code movement only.

Backports commit 7b182eb2467af6c47c9c77c64bbbeed8ed53c330 from qemu
2020-04-30 06:12:26 -04:00
Richard Henderson
a22a2a8b71 target/arm: Introduce core_to_aa64_mmu_idx
If by context we know that we're in AArch64 mode, we need not
test for M-profile when reconstructing the full ARMMMUIdx.

Backports commit 20dc67c947a691fa9df05e76aec6df50204b4b94 from qemu
2020-04-30 05:58:59 -04:00
Richard Henderson
d3a5843aeb target/arm: Replicate TBI/TBID bits for single range regimes
Replicate the single TBI bit from TCR_EL2 and TCR_EL3 so that
we can unconditionally use pointer bit 55 to index into our
composite TBI1:TBI0 field.

Backports commit 3e270f67f0f05277021763af119a6ce195f8ed51 from qemu
2020-04-30 05:58:59 -04:00
Richard Henderson
cc32a96183 target/arm: Honor the HCR_EL2.TTLB bit
This bit traps EL1 access to tlb maintenance insns.

Backports commit 30881b7353b5bb41210c32cd8e00421da757808c from qemu
2020-04-30 05:58:59 -04:00
Richard Henderson
74d6aa6012 target/arm: Honor the HCR_EL2.TPU bit
This bit traps EL1 access to cache maintenance insns that operate
to the point of unification. There are no longer any references to
plain aa64_cacheop_access, so remove it.

Backports commit 38262d8a732f8bd0e9ca3dc064f6e73d00c08b9a from qemu
2020-04-30 05:58:59 -04:00
MerryMage
f9b909f529 header_gen: Add gen_{u,s}shl_i{32,64} to arm 2020-04-13 19:38:59 +01:00
Richard Henderson
f35a83d5ff target/arm: Honor the HCR_EL2.TPCP bit
This bit traps EL1 access to cache maintenance insns that operate
to the point of coherency or persistence.

Backports commit 1bed4d2e55459129c19f5952bcfc65bd0c70db5b from qemu
2020-03-22 02:44:41 -04:00
Richard Henderson
8ff8ff0c4a target/arm: Honor the HCR_EL2.TACR bit
This bit traps EL1 access to the auxiliary control registers.

Backports commit 9960237769ada2faaaf1898b96da7a55e1691cf4 from qemu
2020-03-22 02:42:05 -04:00
Richard Henderson
d252af2069 target/arm: Honor the HCR_EL2.TSW bit
These bits trap EL1 access to set/way cache maintenance insns.

Backports commit 1803d2713b29d85031cc964d545036bda9880f26 from qemu
2020-03-22 02:40:10 -04:00
Richard Henderson
7ee27e5d93 target/arm: Honor the HCR_EL2.{TVM,TRVM} bits
These bits trap EL1 access to various virtual memory controls.

Backports commit 84929218512c19ec9a296fbfd7b39219e0c592ae from qemu
2020-03-22 02:38:03 -04:00
Richard Henderson
7cd75be7c8 target/arm: Improve masking in arm_hcr_el2_eff
Update the {TGE,E2H} == '11' masking to ARMv8.6.
If EL2 is configured for aarch32, disable all of
the bits that are RES0 in aarch32 mode.

Backports commit 4990e1d3c128580dd2fa0bbb1a42b6d63ba1ac28 from qemu
2020-03-22 02:32:35 -04:00
Richard Henderson
fa599a9538 target/arm: Add HCR_EL2 bit definitions from ARMv8.6
Backports commit e0a38bb35aa930c2d3b9982914297f0c0e8fd5c8 from qemu
2020-03-22 02:31:19 -04:00
Richard Henderson
c4b2493c2e target/arm: Improve masking of HCR/HCR2 RES0 bits
Don't merely start with v8.0, handle v7VE as well. Ensure that writes
from aarch32 mode do not change bits in the other half of the register.
Protect reads of aa64 id registers with ARM_FEATURE_AARCH64.

Backports commit d1fb4da208411ce7b3dafb9f9e7726ebcec14edb from qemu
2020-03-22 02:28:41 -04:00
Peter Maydell
32b0e506e6 target/arm: Implement (trivially) ARMv8.2-TTCNP
The ARMv8.2-TTCNP extension allows an implementation to optimize by
sharing TLB entries between multiple cores, provided that software
declares that it's ready to deal with this by setting a CnP bit in
the TTBRn_ELx. It is mandatory from ARMv8.2 onward.

For QEMU's TLB implementation, sharing TLB entries between different
cores would not really benefit us and would be a lot of work to
implement. So we implement this extension in the "trivial" manner:
we allow the guest to set and read back the CnP bit, but don't change
our behaviour (this is an architecturally valid implementation
choice).

The only code path which looks at the TTBRn_ELx values for the
long-descriptor format where the CnP bit is defined is already doing
enough masking to not get confused when the CnP bit at the bottom of
the register is set, so we can simply add a comment noting why we're
relying on that mask.

Backports commit 41a4bf1feab098da4cd5495cd56a99b0339e2275 from qemu
2020-03-22 02:24:48 -04:00
Anup Patel
d47390ade4 target/riscv: Emulate TIME CSRs for privileged mode
Currently, TIME CSRs are emulated only for user-only mode. This
patch add TIME CSRs emulation for privileged mode.

For privileged mode, the TIME CSRs will return value provided
by rdtime callback which is registered by QEMU machine/platform
emulation (i.e. CLINT emulation). If rdtime callback is not
available then the monitor (i.e. OpenSBI) will trap-n-emulate
TIME CSRs in software.

We see 25+% performance improvement in hackbench numbers when
TIME CSRs are not trap-n-emulated.

Backports commit c695724868ce4049fd79c5a509880dbdf171e744 from qemu
2020-03-22 02:22:17 -04:00
Alistair Francis
6c3338430a target/riscv: Add the MSTATUS_MPV_ISSET helper macro
Add a helper macro MSTATUS_MPV_ISSET() which will determine if the
MSTATUS_MPV bit is set for both 32-bit and 64-bit RISC-V.

Backports commit e44b50b5b2e508fdd24915ab0e44ac49685e1de3 from qemu
2020-03-22 02:18:02 -04:00
Alistair Francis
835b025692 target/riscv: Add support for the 32-bit MSTATUSH CSR
Backports commit 551fa7e8a695ea5fd1cca8ffd318556855bbf54f from qemu
2020-03-22 02:16:13 -04:00
Alistair Francis
63d5b37ebc target/riscv: Set htval and mtval2 on execptions
Backports commit 3067553993ae986b76a92df8a978778134ecdc84 from qemu
2020-03-22 02:13:07 -04:00
Alistair Francis
141c47a286 target/riscv: Raise the new execptions when 2nd stage translation fails
Backports commit b2ef6ab9fee6948cf016f9e741feecdfb333fcbe from qemu
2020-03-22 02:08:53 -04:00
Alistair Francis
773757a919 target/riscv: Implement second stage MMU
Backports commit 36a18664bafcfafa5e997b47458387f6fe53d537 from qemu
2020-03-22 02:07:49 -04:00
Alistair Francis
e350bd6f85 target/riscv: Allow specifying MMU stage
Backports commit 1448689c7b23690f49a4cce248c6e4ac973d37b8 from qemu
2020-03-22 01:56:41 -04:00
Alistair Francis
74b8fd51f4 target/riscv: Respect MPRV and SPRV for floating point ops
mark_fs_dirty() is the only place in translate.c that uses the
virt_enabled bool. Let's respect the contents of MSTATUS.MPRV and
HSTATUS.SPRV when setting the bool as this is used for performing
floating point operations when V=0.

Backports commit ae84dd0ab7eaf7e98cd6ee05b2063cce8ff9bc02 from qemu
2020-03-22 01:54:06 -04:00
Alistair Francis
270808e22e target/riscv: Mark both sstatus and msstatus_hs as dirty
Mark both sstatus and vsstatus as dirty (3).

Backports commit 45b4dc8b403aa5473ec015336adf7d14d88e85c5 from qemu
2020-03-22 01:51:41 -04:00
Alistair Francis
347ad87e98 target/riscv: Disable guest FP support based on virtual status
When the Hypervisor extension is in use we only enable floating point
support when both status and vsstatus have enabled floating point
support.

Backports commit 29409c1d921d607873268671bf11a088efb5558e from qemu
2020-03-22 01:49:05 -04:00
Alistair Francis
0d488f3a76 target/riscv: Only set TB flags with FP status if enabled
Backports commit e28eaed87902c5e852c3ed043b27204e879aa4e2 from qemu
2020-03-22 01:45:32 -04:00
Alistair Francis
1e17cbc52c target/riscv: Remove the hret instruction
The hret instruction does not exist in the new spec versions, so remove
it from QEMU.

Backports commit 0736febb2d0e1bb503ca07091c16a16e78480366 from qemu
2020-03-22 01:44:55 -04:00
Alistair Francis
13a7b74827 target/riscv: Add hfence instructions
Backports commit 895c412cb6e79b7b08bd3c2d2fcb70a3cab6ff8a from qemu
2020-03-22 01:43:53 -04:00
Alistair Francis
fcc30eac4d target/riscv: Add Hypervisor trap return support
Backports commit e3fba4bab668a41f7cec6405c5aeb21497bd7c83 from qemu
2020-03-22 01:41:58 -04:00
Alistair Francis
3e3bdf836b target/riscv: Add hypvervisor trap support
Backports commit 5eb9e782f523d2898e2dacd86c6e41365dae74b3 from qemu
2020-03-22 01:40:06 -04:00
Alistair Francis
f6b14e146e target/riscv: Generate illegal instruction on WFI when V=1
Backports commit 9d0d11269671646be7475cc01142e9d3ed8ae59c from qemu
2020-03-22 01:38:14 -04:00
Alistair Francis
4af0dc9a7e target/ricsv: Flush the TLB on virtulisation mode changes
To ensure our TLB isn't out-of-date we flush it on all virt mode
changes. Unlike priv mode this isn't saved in the mmu_idx as all
guests share V=1. The easiest option is just to flush on all changes.

Backports commit eccc5a12c2fd1c646c69a1e7de29183b7a559973 from qemu
2020-03-22 01:37:19 -04:00
Alistair Francis
5adba06c29 target/riscv: Add support for virtual interrupt setting
Backports commit 3ef10a098b0d3ebb02bf8e1325adc3b77af92f0b from qemu
2020-03-22 01:36:28 -04:00
Alistair Francis
4e7aaee31a target/riscv: Extend the SIP CSR to support virtulisation
Backports commit a2e9f57d06279220b1834eca2494e52adae121b8 from qemu
2020-03-22 01:35:11 -04:00
Alistair Francis
1fd49ac6f0 target/riscv: Extend the MIE CSR to support virtulisation
Backports commit d0e53ce33ec8f66ffa597c634d50be73264aeadb from qemu
2020-03-22 01:33:17 -04:00
Alistair Francis
6aabd67ef8 target/riscv: Set VS bits in mideleg for Hyp extension
Backports commit 713d8363deb3774db14fb88a9fcd99687dcef114 from qemu
2020-03-22 01:32:12 -04:00
Alistair Francis
ebc7b9371f target/riscv: Add virtual register swapping function
Backports commit 66e594f2800ddc55f908830bf9e8dc4cda1304fe from qemu
2020-03-22 01:30:22 -04:00
Alistair Francis
042e3df075 target/riscv: Add Hypervisor machine CSRs accesses
Backports commit 34cfb5f61842d495c6f6fc3eeb4197b5b44fd570 from qemu
2020-03-22 01:26:47 -04:00
Alistair Francis
dd356201a6 target/riscv: Add Hypervisor virtual CSRs accesses
Backports commit 8747c9eeb2aaec8441d0900b198725ab33af4951 from qemu
2020-03-22 01:24:42 -04:00
Alistair Francis
221c427fc3 target/riscv: Add Hypervisor CSR access functions
Backports commit ff2cc1294cd8179d87de299b8e7a16bdb1e69523 from qemu
2020-03-22 01:22:58 -04:00
Alistair Francis
e4e41d3d87 target/riscv: Fix CSR perm checking for HS mode
Update the CSR permission checking to work correctly when we are in
HS-mode.

Backports commit 0a42f4c4408824dc7cb9ff60c9bdce6dcc0d24a5 from qemu
2020-03-22 01:19:04 -04:00
Alistair Francis
ab7c1570d8 target/riscv: Add the force HS exception mode
Add a FORCE_HS_EXCEP mode to the RISC-V virtulisation status. This bit
specifies if an exeption should be taken to HS mode no matter the
current delegation status. This is used when an exeption must be taken
to HS mode, such as when handling interrupts.

Backports commit c7b1bbc80fc2af17395d3986c346fd2307e57829 from qemu
2020-03-22 01:16:55 -04:00
Alistair Francis
95537388c5 target/riscv: Add the virtulisation mode
Backports commit ef6bb7b62682badefdcb744831510aaa5971684f from qemu
2020-03-22 01:15:06 -04:00
Alistair Francis
fa04e7ea45 target/riscv: Rename the H irqs to VS irqs
Backports commit 205377f8940898e4c53d1b44350a3d4934a2da72 from qemu
2020-03-22 01:09:04 -04:00
Alistair Francis
0198a09698 target/riscv: Add support for the new execption numbers
The v0.5 Hypervisor spec add new execption numbers, let's add support
for those.

Backports commit ab67a1d07a4f6f1b4d577c5c47013273b9804551 from qemu
2020-03-22 01:07:23 -04:00
Alistair Francis
870603305d target/riscv: Add the Hypervisor CSRs to CPUState
Add the Hypervisor CSRs to CPUState and at the same time (to avoid
bisect issues) update the CSR macros for the v0.5 Hyp spec.

Backports commit bd023ce33b85d73791b7bc78fd04a8115c60995e from qemu
2020-03-22 01:05:23 -04:00
Alistair Francis
40afe12008 target/riscv: Add the Hypervisor extension
Backports commit af1fa0039c799a350bcde07b3d8a71dfde07d11b from qemu
2020-03-22 01:01:07 -04:00
Alistair Francis
83ac8747a5 target/riscv: Convert MIP CSR to target_ulong
The MIP CSR is a xlen CSR, it was only 32-bits to allow atomic access.
Now that we don't use atomics for MIP we can change this back to a xlen
CSR.

Backports commit 028616130d5f0abc8a3b96f28963da51a875024b from qemu
2020-03-22 00:59:47 -04:00
Peter Maydell
7271ebf96d target/arm: Implement ARMv8.3-CCIDX
The ARMv8.3-CCIDX extension makes the CCSIDR_EL1 system ID registers
have a format that uses the full 64 bit width of the register, and
adds a new CCSIDR2 register so AArch32 can get at the high 32 bits.

QEMU doesn't implement caches, so we just treat these ID registers as
opaque values that are set to the correct constant values for each
CPU. The only thing we need to do is allow 64-bit values in our
cssidr[] array and provide the CCSIDR2 accessors.

We don't set the CCIDX field in our 'max' CPU because the CCSIDR
constant values we use are the same as the ones used by the
Cortex-A57 and they are in the old 32-bit format. This means
that the extra regdef added here is unused currently, but it
means that whenever in the future we add a CPU that does need
the new 64-bit format it will just work when we set the cssidr
values and the ID registers for it.

Backports commit 957e615503bd0de22393fd8dbcb22a5064fd2b5c from qemu
2020-03-22 00:17:37 -04:00
Peter Maydell
5416c5a672 target/arm: Implement v8.4-RCPC
The v8.4-RCPC extension implements some new instructions:
* LDAPUR, LDAPURB, LDAPURH, LDAPRSB, LDAPRSH, LDAPRSW
* STLUR, STLURB, STLURH

These are all in a new subgroup of encodings that sits below the
top-level "Loads and Stores" group in the Arm ARM.

The STLUR* instructions have standard store-release semantics; the
LDAPUR* have Load-AcquirePC semantics, but (as with LDAPR*) we choose
to implement them as the slightly stronger Load-Acquire.

Backports commit a1229109dec4375259d3fff99f362405aab7917a from qemu
2020-03-22 00:15:46 -04:00
Peter Maydell
f72582bb7a target/arm: Implement v8.3-RCPC
The v8.3-RCPC extension implements three new load instructions
which provide slightly weaker consistency guarantees than the
existing load-acquire operations. For QEMU we choose to simply
implement them with a full LDAQ barrier.

Backports commit 2677cf9f92a5319bb995927f9225940414ce879d from qemu
2020-03-22 00:13:08 -04:00