summaryrefslogtreecommitdiff
path: root/sys/arch
AgeCommit message (Collapse)Author
2024-04-19Fix typo in commentJeremie Courreges-Anglas
2024-04-19Revert per-CPU caches a double-free has been found by naddy@.Martin Pieuchot
2024-04-17Add per-CPU caches to the pmemrange allocator.Martin Pieuchot
The caches are used primarily to reduce contention on uvm_lock_fpageq() during concurrent page faults. For the moment only uvm_pagealloc() tries to get a page from the current CPU's cache. So on some architectures the caches are also used by the pmap layer. Each cache is composed of two magazines, design is borrowed from jeff bonwick vmem's paper and the implementation is similar to the one of pool_cache from dlg@. However there is no depot layer and magazines are refilled directly by the pmemrange allocator. Tested by robert@, claudio@ and Laurence Tratt. ok kettenis@
2024-04-14Turn sp_tlb_flush_{ctx,pte} into function pointers, and pick one out of threeMiod Vallat
flavours: pre-usIII, usIII, and sun4v. This allows us to get rid of the HORRID_III_HACK define in locore and switch pre-usIII systems to the older, slightly simpler, code for these routines. ok claudio@ kettenis@
2024-04-14Implement support for AVX-512. This required some fixes to the so-farMark Kettenis
unused Skylake AVX-512 MDS handler and increases the ci_mds_tmp array to 64 bytes. With help from guenther@ ok deraadt@, guenther@
2024-04-14with empty body loops, put final semicolon on a new line for readabilityJonathan Gray
ok bluhm@ jca@
2024-04-13correct indentationJonathan Gray
no functional change, found by smatch warnings ok miod@ bluhm@
2024-04-13vmm: protect vmm activation with the vmm_softc rwlock.Dave Voutila
Syzbot found a race when enabling vmm mode on multiprocessor systems. Protect the vmm start/stop lifecycle by taking the write lock used for protecting the status of the vmm device. Reported-by: syzbot+6ae9cec00bbe45fd7782@syzkaller.appspotmail.com ok gnezdo@
2024-04-13Split out Spectre-V2 and Spectre-BHB mitigation code like I did forMark Kettenis
Spectre-V4 a few weeks ago. Treat Qualcomm Kryo 400 Silver like Cortex-A55 for Spectre-V2 since that is what is is. ok jsg@
2024-04-11correct value of XFEATURE_AMXJonathan Gray
ok miod@ guenther@
2024-04-11Stop making <machine/pmap.h> include <machine/pte.h>, and fix the very fewMiod Vallat
files which really need <machine/pte.h> guts.
2024-04-10Use km_alloc(9) to allocate USPACE instead of uvm_pglistalloc(9).Martin Pieuchot
ok miod@
2024-04-09vmm/vmd: add exception injection and refactor inject api.Dave Voutila
In order to continue work on mmio and other instruction emulation, vmd(8) needs the ability to inject exceptions (like page faults) from userland. Refactor the way events are injected from userland, cleaning up how hardware (external) interrupts are injected in the process. ok mlarkin@
2024-04-08Stop redeclaring ENTRY, the <machine/asm.h> flavour is as good as we need it.Miod Vallat
While there, replace inlined NENTRY by actual use of that macro. ok kettenis@
2024-04-08Fix misplaced END() macro.Miod Vallat
ok kettenis@
2024-04-08Remove unused tf_fault and tf_kstack members from struct trapframe. TheseMiod Vallat
fields were (seldom) written to but never used for anything. ok kettenis@
2024-04-08Remove demap code for MMU miss faults.Miod Vallat
This code was #if 0, except for instruction misses where it had been enabled probably by mistake... and was demapping in the data mmu anyway... (#include <facepalm.h>) ok kettenis@
2024-04-08There is too much #ifdef DEBUG stuff cluttering locore, really. While some ofMiod Vallat
it had sense in the early days of the sparc64 port, this code has bitrotten and is getting in the way. Time for a visit to the Attic. This removes: - interrupt handling debug code (forcing hz = 1, probably broken since years). - unused or too invasive DEBUG code which noone will ever use in this state. - #if 0 code blocks which have been this way since locore.s revision 1.1 and will never get enabled. ok kettenis@
2024-04-08Introduce more assembler macros to deduplicate code and improve readability:Miod Vallat
- one macro for the inline pseg_get logic used in various MMU trap handlers. - one macro for the TSB locking logic in various PTE update routines. - one macro for the sun4v rwindow content saving. ok kettenis@
2024-04-08Remove dead code and data, fix typos, kill wrong comments.Miod Vallat
ok kettenis@
2024-04-08Fix cut'n'paste error in data miss handler causing wrong label to be used inMiod Vallat
some failure conditions.
2024-04-08Fix sun4v patching of sp_tlb_flush_ctx() to pass flags in the right register.Miod Vallat
ok kettenis@
2024-04-07The RISC-V architecture has cache-coherent DMA... until it doesn't. ThisMark Kettenis
is indicated by a "dma-noncoherent" property on the bus or device nodes in the device tree. Set the BUS_DMA_COHERENT flag on the DMA tag for mainbus(4) and modify the flags based on the presence of "dma-coherent" and "dma-noncoherent" properties where appropriate. ok patrick@
2024-04-06Now that we support RISC-V CPUs that have MMUs with memory cachabilityMark Kettenis
attributes, the "direct map" becomes problematic as it results in mappings for the same physical memory pages with different cachability addresses. The RISC-V specification of the "Svpbmt" extension doesn't outright state that this is "verboten" like on some other architectures that we support. But it does say that it may result in access with the wrong attributes. So restrict the use of the direct map to just mapping the 64MB block that the bootloader loaded us into. To make this possible map the device tree later like we do on arm64. This allows us to get rid of some assembly code in locore.S as a bonus! ok miod@, jca@
2024-04-03Initialize earlier macppc's and powerpc64's pmap_hash_lockGeorge Koehler
At boot, the powerpc64 kernel was calling pmap_bootstrap -> pmap_kenter_pa -> mtx_enter(&pmap_hash_lock) before it did pmap_init -> mtx_init(&pmap_hash_lock, IPL_HIGH) Change from mtx_init to MUTEX_INITIALIZER. This allows an option WITNESS kernel to boot without warning of an uninitialized mutex. Also change macppc's pmap_hash_lock from __ppc_lock_init to PPC_LOCK_INITIALIZER, though WITNESS doesn't see this lock. ok mpi@
2024-04-03pmap_virtual_space() and pmap_steal_memory() are mutually exclusive, soMiod Vallat
make sure only one of them is prototyped and only one of them is implemented. ok mpi@ kettenis@
2024-04-03Add ci_cpuid_level and ci_vendor holding the per-CPU basic cpuidPhilip Guenther
level and a numeric mapping of the cpu vendor, both from CPUID(0). Convert the general use of strcmp(cpu_vendor) to simple numeric tests of ci_vendor. Track the minimum of all ci_cpuid_level in the cpuid_level global and continue to use that for what we vmm exposes. AMD testing help matthieu@ krw@ ok miod@ deraadt@ cheloha@
2024-04-01Delete 108 lines of ASM from vmx_enter_guest() that predated lotsPhilip Guenther
of later enhancements, removing the save/restore of flags, selectors, and MSRs: flags are caller-saved and don't need restoring while selectors and MSRs are auto-restored. The FSBASE, GSBASE, and KERNELGSBASE MSRs just need the correct values set with vmwrite() in the "on new CPU?" block of vcpu_run_vmx(). Also, only rdmsr(MSR_MISC_ENABLE) once in vcpu_reset_regs_vmx(), give symbolic names to the exit-load MSR slots, eliminate VMX_NUM_MSR_STORE, and #if 0 the vc_vmx_msr_entry_load_{va,pa} code and definitions as unused. ok dv@
2024-03-31Enable rge(4).Mark Kettenis
2024-03-31Remove long dead isa-on-powerpc leftovers.Miod Vallat
2024-03-31Stop referring to dead-since-30-years gatherstats() in comments.Miod Vallat
2024-03-30Use illop1 as failure instruction in the locore gap file.Miod Vallat
2024-03-30Bring back an inline db_enter(), which makes backtraces easier to grokMiod Vallat
when sh*t hits the fan; per kettenis@ request and forgotten in previous cleaning commit.
2024-03-30use void in function decl with no args; avoids non-ANSI smatch warningsJonathan Gray
2024-03-29Use SBI calls to reboot or power down the machine when the firmwareMark Kettenis
supports them. ok jca@
2024-03-29Drop the first argument of intr_establish().Miod Vallat
Instead, require all callers to put the right value in the ih_pil field, and have intr_establish() trust them rather than assigning this field again from its first argument. ok claudio@ kettenis@
2024-03-29Store the physical address of each pcb in struct mdproc, and use this inMiod Vallat
order to speed up window spills, rather than doing an inline pmap_extract (well, pseg_get). ok claudio@ kettenis@
2024-03-29Remove truly unneeded includes (not included indirectly).Miod Vallat
ok claudio@ kettenis@
2024-03-29Check for %otherwin being nonzero earlier in rft_user.Miod Vallat
There is one code path using it in %g2 and another using it in %g7. There is no reason for them to use different registers, and fixing this allows the check to be performed a bit earlier. ok claudio@ kettenis@
2024-03-29Simplify two more places where we don't need to check for unbiased stacks.Miod Vallat
ok claudio@ kettenis@
2024-03-29STACKFRAME macro is always invoked on a well-formed 64-bit stack, no needMiod Vallat
to check for missing BIAS. ok claudio@ kettenis@
2024-03-29Text faults should not invoke uvm_grow() since the fault address is not onMiod Vallat
stack. Remove duplicated "panic if uvm_fault() fails and we are in kernel mode" blocks. ok claudio@ kettenis@
2024-03-29send_softint() was designed as being able to target a particular cpu, but theMiod Vallat
code for this was never written and all uses target the running cpu anyway, so stop pretending it may do things it won't do and drop that cpu argument. ok claudio@ kettenis@
2024-03-29Update/fix/remove obsolete or just plainly wrong comments.Miod Vallat
ok claudio@ kettenis@
2024-03-29Stop including <machine/signal.h> in locore and remove _LOCORE guardsMiod Vallat
from this header file. ok claudio@ kettenis@
2024-03-29Move C declarations from intreg.h to <machine/intr.h> which includes it.Miod Vallat
This makes intreg.h locore-friendly - it only contains the MAXINTNUM define after that. ok claudio@ kettenis@
2024-03-29Get rid of unnecessary defines in assym.h.Miod Vallat
ok claudio@ kettenis@
2024-03-29Clean the fpu trap code:Miod Vallat
- since there are no hardware fpu operation queues on real sparc64 hardware, don't bother declaring the relevant struct and fields. - when an fpu instruction needs to be emulated, pass it directly to fpu_cleanup rather than fake its appearance in the fpu queue. While there, also pass the ready-to-use union sigval computed in trap() in case a signal needs to be delivered. ok claudio@ kettenis@
2024-03-29Be sure to reset p->p_md.md_fpstate to NULL after freeing it in cpu_exit().Miod Vallat
There should hopefully be no further faults on this proc causing an fpu state to be handled, but better play safe than sorry. ok claudio@ kettenis@
2024-03-29Clean-up cache-related definitions and routines.Miod Vallat
ok claudio@ kettenis@
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-26 11:37:32 +0000