Commit Graph

72 Commits

Author SHA1 Message Date
Richard Henderson
2a4a7b9391
tcg: Use tlb_fill probe from tlb_vaddr_to_host
Most of the existing users would continue around a loop which
would fault the tlb entry in via a normal load/store.

But for AArch64 SVE we have an existing emulation bug wherein we
would mark the first element of a no-fault vector load as faulted
(within the FFR, not via exception) just because we did not have
its address in the TLB. Now we can properly only mark it as faulted
if there really is no valid, readable translation, while still not
raising an exception. (Note that beyond the first element of the
vector, the hardware may report a fault for any reason whatsoever;
with at least one element loaded, forward progress is guaranteed.)

Backports commit 4811e9095c0491bc6f5450e5012c9c4796b9e59d from qemu
2019-05-16 18:27:03 -04:00
Lioncash
3521e72580
target/arm: Sychronize with qemu
Synchronizes with bits and pieces that were missed due to merging
incorrectly (sorry :<)
2019-04-18 04:49:11 -04:00
Lioncash
5f12065284
sve_helper: Use the QEMU_FLATTEN macro instead of the compiler attribute directly
Keeps the code compiler-independent.
2018-10-23 13:05:02 -04:00
Richard Henderson
66ffb372e7
target/arm: Pass TCGMemOpIdx to sve memory helpers
There is quite a lot of code required to compute cpu_mem_index,
or even put together the full TCGMemOpIdx. This can easily be
done at translation time.

Backports commit 500d04843ba953dc4560e44f04001efec38c14a6 from qemu
2018-10-08 14:15:15 -04:00
Richard Henderson
606e8cdb8c
target/arm: Rewrite vector gather first-fault loads
This implements the feature for softmmu, and moves the
main loop out of a macro and into a function.

Backports commit 116347ce20bb7b5cac17bf2b0e6f607530b50862 from qemu
2018-10-08 14:15:15 -04:00
Richard Henderson
1cd3c2a408
target/arm: Split contiguous stores for endianness
We can choose the endianness at translation time, rather than
re-computing it at execution time.

Backports commit 28d57f2dc59c287e1c40239509b0a325fd00e32f from qemu
2018-10-08 14:15:15 -04:00
Richard Henderson
c9569b3fe0
target/arm: Split contiguous loads for endianness
We can choose the endianness at translation time, rather than
re-computing it at execution time.

Backports commit 7d0a57a2e1cea188b9023261a404d7a211117230 from qemu
2018-10-08 14:15:15 -04:00
Richard Henderson
2542ad17d0
target/arm: Rewrite vector gather stores
This fixes the endianness problem for softmmu, and moves
the main loop out of a macro and into an inlined function.

Backports commit 78cf1b886aa1b95c97fc5114641515c2892bb240 from qemu
2018-10-08 14:15:15 -04:00
Richard Henderson
ff63807164
target/arm: Rewrite vector gather loads
This fixes the endianness problem for softmmu, and moves
the main loop out of a macro and into an inlined function.

Backports commit d4f75f25b43041e7a46d12352b3c70ae457d8cea from qemu
2018-10-08 14:15:15 -04:00
Richard Henderson
966ea163a3
target/arm: Rewrite helper_sve_st[1234]*_r
This fixes the endianness problem for softmmu, and moves the
main loop out of a macro and into an inlined function

Backports commit 9fd46c8362e0a45d04ccceae7051d06dd65c1d57 from qemu
2018-10-08 14:15:15 -04:00
Richard Henderson
4978d77039
target/arm: Rewrite helper_sve_ld[234]*_r
Use the same *_tlb primitives as we use for ld1.

For linux-user, this hoists the set of helper_retaddr. For softmmu,
hoists the computation of the current mmu_idx outside the loop,
fixes the endianness problem, and moves the main loop out of a
macro and into an inlined function.

Backports commit f27d4dc2af0de9b7b45c955882b8420905c6efe8 from qemu
2018-10-08 14:15:15 -04:00
Richard Henderson
5b88176e1d
target/arm: Rewrite helper_sve_ld1*_r using pages
Uses tlb_vaddr_to_host for correct operation with softmmu.
Optimize for accesses within a single page or pair of pages.

Backports commit 9123aeb6fcb14e0955ebe4e2a613802cfa0503ea from qemu
2018-10-08 14:15:15 -04:00
Richard Henderson
118495f4b1
target/arm: Use fp_status_fp16 for do_fmpa_zpzzz_h
This makes float16_muladd correctly use FZ16 not FZ.

Fixes: 6ceabaad110

Backports commit 52a339b11d1719a6589de40606859939875fda9a from qemu
2018-08-17 14:04:20 -04:00
Richard Henderson
1ca7c30fbb
target/arm: Fix typo in helper_sve_ld1hss_r
Backports commit 573ec0fe40b9a412085ac7dfb41975a0fc2b28dd from qemu
2018-08-17 13:47:38 -04:00
Richard Henderson
e2e7bb0e21
target/arm: Fix typo in helper_sve_movz_d
Backports commit 054e7adf4e64e4acb3b033348ebf7cc871baa34f from qemu
2018-08-16 07:12:18 -04:00
Richard Henderson
f26356b930
target/arm: Reorganize SVE WHILE
The pseudocode for this operation is an increment + compare loop,
so comparing <= the maximum integer produces an all-true predicate.

Rather than bound in both the inline code and the helper, pass the
helper the number of predicate bits to set instead of the number
of predicate elements to set.

Backports commit bbd0968c458d48e34a08b8694fa3309a9fe1c9e7 from qemu
2018-08-16 07:09:33 -04:00
Richard Henderson
46fd2c485a
target/arm: Fix sign of sve_cmpeq_ppzw/sve_cmpne_ppzw
The normal vector element is sign-extended before
comparing with the wide vector element.

Backports commit df4e001093988544d09887122ae824f18ba55c68 from qemu
2018-08-16 07:04:52 -04:00
Richard Henderson
bd9975a8ab
target/arm: Fix LD1W and LDFF1W (scalar plus vector)
'I' was being double-incremented; correctly within the inner loop
and incorrectly within the outer loop.

Backports commit 628fc75f3a3bb115de3b445c1a18547c44613cfe from qemu
2018-07-17 12:33:00 -04:00
Richard Henderson
f64e48dbee
target/arm: Fix SVE signed division vs x86 overflow exception
We already check for the same condition within the normal integer
sdiv and sdiv64 helpers. Use a slightly different formation that
does not require deducing the expression type.

Backports commit 7e8fafbfd0537937ba8fb366a90ea6548cc31576 from qemu
2018-07-03 05:06:06 -04:00
Richard Henderson
63431f0c21
target/arm: Implement SVE fp complex multiply add
Backports commit 05f48bab3080fb876fbad8d8f14e6ba545432d67 from qemu
2018-07-03 04:21:41 -04:00
Richard Henderson
79220741df
target/arm: Implement SVE floating-point complex add
Backports commit 76a9d9cdc481ed79f1c2ec16eeed185f13e6a8ae from qemu
2018-07-03 04:17:41 -04:00
Richard Henderson
00d3671412
target/arm: Implement SVE floating-point unary operations
Backports commit ec5b375bb5a0e35c0c21dc9dd1d82894269ce215 from qemu
2018-07-03 04:11:12 -04:00
Richard Henderson
2c2c940ef8
target/arm: Implement SVE floating-point round to integral value
Backports commit cda3c75322c6fae1cc5b367ee6d7acf2cbdbcf2b from qemu
2018-07-03 04:07:21 -04:00
Richard Henderson
a9160a0e08
target/arm: Implement SVE floating-point convert to integer
Backports commit df4de1affc440d6f2cdaeea329b90c0b88ece5a1 from qemu
2018-07-03 04:02:58 -04:00
Richard Henderson
524630de27
target/arm: Implement SVE floating-point convert precision
Backports commit 46d33d1e3c9c5d56d57056db55010de52c173902 from qemu
2018-07-03 03:58:07 -04:00
Richard Henderson
b14793a933
target/arm: Implement SVE floating-point trig multiply-add coefficient
Backports commit 67fcd9ad35d2b38630ee34e8ced8878d334c74fb from qemu
2018-07-03 03:52:36 -04:00
Richard Henderson
f7e0c9e079
target/arm: Implement SVE FP Compare with Zero Group
Backports commit 4d2e2a03384a43c641e0cbca7ac79d7d0c50f666 from qemu
2018-07-03 03:49:52 -04:00
Richard Henderson
f9f228efec
target/arm: Implement SVE FP Fast Reduction Group
Backports commit 23fbe79faa38cb4acc59f956a63feba3c2cc73ac from qemu
2018-07-03 03:41:00 -04:00
Richard Henderson
c718ef4243
target/arm: Implement SVE floating-point arithmetic with immediate
Backports commit cc48affe83fff4b2886c064265d7103dee5e4a14 from qemu
2018-07-03 03:24:46 -04:00
Richard Henderson
db1d39ab4a
target/arm: Implement SVE floating-point compare vectors
Backports commit abfdefd5bd444b629d16dcefc2b60ac8da37e87d from qemu
2018-07-03 03:14:24 -04:00
Richard Henderson
892a9a66eb
target/arm: Implement SVE first-fault gather loads
Backports commit ed67eb7fa2a63b6709ec94397d833bc3686f7833 from qemu
2018-07-03 03:06:07 -04:00
Richard Henderson
1782f20dde
target/arm: Implement SVE gather loads
Backports commit 673e9fa6c29e030f4ab6ceae5d0f50bd36fe0ee0 from qemu
2018-07-03 02:58:18 -04:00
Richard Henderson
b78e283513
target/arm: Implement SVE scatter stores
Backports commit f6dbf62a7e3d00e9a1dcc7fe3e53b32c3ed93e24 from qemu
2018-07-03 02:40:17 -04:00
Richard Henderson
c497dc0a83
target/arm: Implement SVE load and broadcast element
Backports commit 684598640dc3b28f86ccc28cc9af50ba257f4cc8 from qemu
2018-07-03 02:27:00 -04:00
Richard Henderson
2caa99929e
target/arm: Implement SVE Floating Point Accumulating Reduction Group
Backports commit 7f9ddf64d5fe5bfaa91ae0ec52217d86f4d86452 from qemu
2018-07-03 02:21:41 -04:00
Richard Henderson
44d89b4cb1
target/arm: Implement SVE FP Multiply-Add Group
Backports commit 6ceabaad1101b0b33490b0fd4bed5b6445b0a34e from qemu
2018-07-03 02:17:16 -04:00
Richard Henderson
d1c5d5b728
target/arm: Implement SVE floating-point arithmetic (predicated)
Backports commit ec3b87c28eb120b6575cc1ed7bfbfbf1b0060163 from qemu
2018-07-03 02:13:04 -04:00
Richard Henderson
e7f53b6d1b
target/arm: Implement SVE integer convert to floating-point
Backports commit 8092c6a314c9625d8976fb6c5b6969f2a1105cd8 from qemu
2018-07-03 02:05:50 -04:00
Richard Henderson
3a51861c05
target/arm: Implement SVE Memory Contiguous Store Group
Backports commit 1a039c7e58533d5f9431a2064771113d5961c964 from qemu
2018-07-03 01:57:52 -04:00
Richard Henderson
22a155a5d4
target/arm: Implement SVE Contiguous Load, first-fault and no-fault
Backports commit e2654d757598d6c06d1ceb25c62ddf7d63cac32f from qemu
2018-07-03 01:52:38 -04:00
Richard Henderson
da1eeb226e
target/arm: Implement SVE Memory Contiguous Load Group
Backports commit c4e7c493da2fdd2c92a7958e592e38200af2f1ba from qemu
2018-07-03 01:40:18 -04:00
Richard Henderson
e56db70fee
target/arm: Implement SVE Integer Wide Immediate - Unpredicated Group
Backports commit 6e6a157d683058d86a224cf128320d67b0915365 from qemu
2018-06-15 14:05:11 -04:00
Richard Henderson
496bb35c97
target/arm: Implement SVE Integer Compare - Scalars Group
Backports commit caf1cefc72be98497e0907d0e07f4327fc641e96 from qemu
2018-06-15 13:54:05 -04:00
Richard Henderson
bc55b3e570
target/arm: Implement SVE Predicate Count Group
Backports commit 9ee3a611de28b8d0862fa687215b04b5aad20747 from qemu
2018-06-15 13:49:58 -04:00
Richard Henderson
bb930f35b0
target/arm: Implement SVE Partition Break Group
Backports commit 35da316f5e847292ffbe7b6d16cd3988043dfe22 from qemu
2018-06-15 13:42:35 -04:00
Richard Henderson
ade246e87b
target/arm: Implement SVE Integer Compare - Immediate Group
Backports commit 38cadeba0daf0f16cf2aeaa5b2752b26fb0676c5 from qemu
2018-06-15 13:35:40 -04:00
Richard Henderson
2969a38d61
target/arm: Implement SVE Integer Compare - Vectors Group
Backports commit 757f9cff1b63895bfd6fc8d66a6e52d7c40baa7b from qemu
2018-06-15 13:29:15 -04:00
Richard Henderson
7211d415a4
target/arm: Implement SVE Select Vectors Group
Backports commit d3fe4a29d754dee73cbf3cb7584db222981179ac from qemu
2018-06-15 13:17:47 -04:00
Richard Henderson
7698c1634e
target/arm: Implement SVE vector splice (predicated)
Backports commit b48ff24098c72f86e187e6abb7e9ca4de40a7fb4 from qemu
2018-06-15 13:14:33 -04:00
Richard Henderson
7d930e8515
target/arm: Implement SVE reverse within elements
Backports commit dae8fb9019d2aa6ccb151a19871df40de6c98e29 from qemu
2018-06-15 13:12:14 -04:00