Make this stuff a bit easier to maintain by using the
root level variables to build up the Toolchain paths.
Also leave a note for future editors of BuildIt.sh to
give them warning about the other changes they'll need
to make.
Until we get the goodness that C++ modules are supposed to be, let's try
to shave off some parse time using precompiled headers.
This commit only adds some very common AK headers, only to binaries,
libraries and the kernel (tests are not covered due to incompatibility
with AK/TestSuite.h).
This option is on by default, but can be disabled by passing
`-DPRECOMPILE_COMMON_HEADERS=OFF` to cmake, which will disable all
header precompilations.
This makes the build about 30 seconds faster on my machine (about 7%).
This enables building usermode programs with exception handling. It also
builds a libstdc++ without exception support for the kernel.
This is necessary because the libstdc++ that gets built is different
when exceptions are enabled. Using the same library binary would
require extensive stubs for exception-related functionality in the
kernel.
This is a very basic implementation that only requests 4096 bytes
of entropy from the host once, but its still high quality entropy
so it should be a good fix for #4490 (boot-time entropy starvation)
for virtualized environments.
Co-authored-by: Sahan <sahan.h.fernando@gmail.com>
This allows converting a single virtual buffer into its non-physically
contiguous parts, this is especially useful for DMA-based devices that
support scatter/gather-like functionality, as it eliminates the need
to clone outgoing buffers into one physically contiguous buffer.
This patch allocates a physical page for each of the virtqueues and
memcpys to it when receiving a buffer to get a physical, aligned
contiguous buffer as required by the virtio specification.
Co-authored-by: Sahan <sahan.h.fernando@gmail.com>
Based on pull #3236 by tomuta, this is still very much WIP but will
eventually allow us to switch from the considerably slower method of
knocking on port 0xe9 for each character
Co-authored-by: Tom <tomut@yahoo.com>
Co-authored-by: Sahan <sahan.h.fernando@gmail.com>
Based on pull #3236 by tomuta, this adds helper methods for generic
device initialization, and partily-broken virtqueue helper methods
Co-authored-by: Tom <tomut@yahoo.com>
Co-authored-by: Sahan <sahan.h.fernando@gmail.com>
`-Wnull-dereference` has found a lot of "possible null dereferences" in userland.
However, in the kernel, no warnings occurred. To keep it that way,
preemptivly add the flag here and remove it once it is enabled system
wide.
This flag makes the compiler check statically for a null deref. It does
not take into account any human-imposed invariants and as such, may need a
`VERIFY(ptr);`, `if(ptr)`, or `if(!ptr)` before using a pointer.
However, as long as a pointer is not reassigned,
the verify will be valid, meaning that adding `VERIFY` can be done sparingly.
Now the kernel supports 2 ECAM access methods.
MMIOAccess was renamed to WindowedMMIOAccess and is what we had until
now - each device that is detected on boot is assigned to a
memory-mapped window, so IO operations on multiple devices can occur
simultaneously due to creating multiple virtual mappings, hence the name
is a memory-mapped window.
This commit adds a new class called MMIOAccess (not to be confused with
the old MMIOAccess class). This class creates one memory-mapped window.
On each IO operation on a configuration space of a device, it maps the
requested PCI bus region to that window. Therefore it holds a SpinLock
during the operation to ensure that no other PCI bus region was mapped
during the call.
A user can choose to either use PCI ECAM with memory-mapped window
for each device, or for an entire bus. By default, the kernel prefers to
map the entire PCI bus region.
The end goal of this commit is to allow to boot on bare metal with no
PS/2 device connected to the system. It turned out that the original
code relied on the existence of the PS/2 keyboard, so VirtualConsole
called it even though ACPI indicated the there's no i8042 controller on
my real machine because I didn't plug any PS/2 device.
The code is much more flexible, so adding HID support for other type of
hardware (e.g. USB HID) could be much simpler.
Briefly describing the change, we have a new singleton called
HIDManagement, which is responsible to initialize the i8042 controller
if exists, and to enumerate its devices. I also abstracted a bit
things, so now every Human interface device is represented with the
HIDDevice class. Then, there are 2 types of it - the MouseDevice and
KeyboardDevice classes; both are responsible to handle the interface in
the DevFS.
PS2KeyboardDevice, PS2MouseDevice and VMWareMouseDevice classes are
responsible for handling the hardware-specific interface they are
assigned to. Therefore, they are inheriting from the IRQHandler class.
If the user requests to force PIO mode, we just create IDEChannel
objects which are capable of sending PIO commands only.
However, if the user doesn't force PIO mode, we create BMIDEChannel
objects, which are sending DMA commands.
This change is somewhat simplifying the code, so each class is
supporting its type of operation - PIO or DMA. The PATADiskDevice
should not care if DMA is enabled or not.
Later on, we could write an IDEChannel class for UDMA modes,
that are available and documented on Intel specifications for their IDE
controllers.
We can't use deferred functions for anything that may require preemption,
such as copying from/to user or accessing the disk. For those purposes
we should use a work queue, which is essentially a kernel thread that
may be preempted or blocked.
Alot of code is shared between i386/i686/x86 and x86_64
and a lot probably will be used for compatability modes.
So we start by moving the headers into one Directory.
We will probalby be able to move some cpp files aswell.
This returns ENOSYS if you are running in the real kernel, and some
other result if you are running in UserspaceEmulator.
There are other ways we could check if we're inside an emulator, but
it seemed easier to just ask. :^)
The hierarchy is AHCIController, AHCIPortHandler, AHCIPort and
SATADiskDevice. Each AHCIController has at least one AHCIPortHandler.
An AHCIPortHandler is an interrupt handler that takes care of
enumeration of handled AHCI ports when an interrupt occurs. Each
AHCIPort takes care of one SATADiskDevice, and later on we can add
support for Port multiplier.
When we implement support of Message signalled interrupts, we can spawn
many AHCIPortHandlers, and allow each one of them to be responsible for
a set of AHCIPorts.
For some reason I don't yet understand, building the kernel with -O2
produces a way-too-large kernel on some people's systems.
Since there are some really nice performance benefits from -O2 in
userspace, let's do a compromise and build Userland with -O2 but
put Kernel back into the -Os box for now.
Due to the non-standard way the boot assembler code is linked into
the kernel (not and actual dependency, but linked via linker.ld script)
both make and ninja weren't re-linking the kernel when boot.S was
changed. This should theoretically work since we use the cmake
`add_dependencies(..)` directive to express a manual dependency
on boot from Kernel, but something is obviously broken in cmake.
We can work around that with a hack, which forces a dependency on
a file we know will always exist in the kernel (init.cpp). So if
boot.S is rebuilt, then init.cpp is forced to be rebuilt, and then
we re-link the kernel. init.cpp is also relatively small, so it
compiles fast.
With the kernel command line issue fixed, we can now enable these
KUBSAN options without getting triple faults on startup:
* alignment
* null
* pointer-overflow
Clangd (CLion) was choking on some of the -fsanitize options, and since
we're not building the kernel with Clang anyway, let's just disable
the options for non-GCC compilers for now.
This removes some hard references to the toolchain, some unnecessary
uses of an external install command, and disables a -Werror flag (for
the time being) - only if run inside serenity.
With this, we can build and link the kernel :^)
KASAN is a dynamic analysis tool that finds memory errors. It focuses
mostly on finding use-after-free and out-of-bound read/writes bugs.
KASAN works by allocating a "shadow memory" region which is used to store
whether each byte of memory is safe to access. The compiler then instruments
the kernel code and a check is inserted which validates the state of the
shadow memory region on every memory access (load or store).
To fully integrate KASAN into the SerenityOS kernel we need to:
a) Implement the KASAN interface to intercept the injected loads/stores.
void __asan_load*(address);
void __asan_store(address);
b) Setup KASAN region and determine the shadow memory offset + translation.
This might be challenging since Serenity is only 32bit at this time.
Ex: Linux implements kernel address -> shadow address translation like:
static inline void *kasan_mem_to_shadow(const void *addr)
{
return ((unsigned long)addr >> KASAN_SHADOW_SCALE_SHIFT)
+ KASAN_SHADOW_OFFSET;
}
c) Integrating KASAN with Kernel allocators.
The kernel allocators need to be taught how to record allocation state
in the shadow memory region.
This commit only implements the initial steps of this long process:
- A new (default OFF) CMake build flag `ENABLE_KERNEL_ADDRESS_SANITIZER`
- Stubs out enough of the KASAN interface to allow the Kernel to link clean.
Currently the KASAN kernel crashes on boot (triple fault because of the crash
in strlen other sanitizer are seeing) but the goal here is to just get started,
and this should help others jump in and continue making progress on KASAN.
References:
* ASAN Paper: https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/37752.pdf
* KASAN Docs: https://github.com/google/kasan
* NetBSD KASAN Blog: https://blog.netbsd.org/tnf/entry/kernel_address_sanitizer_part_3
* LWN KASAN Article: https://lwn.net/Articles/612153/
* Tracking Issue #5351
Let's be a little more expressive when inducing a kernel panic. :^)
PANIC(...) passes any arguments you give it to dmesgln(), then prints
a backtrace and hangs the machine.
CLion doesn't understand that we switch compilers mid-build (which I
can understand since it's a bit unusual.) Defining __serenity__ makes
the majority of IDE features work correctly in the kernel context.
This patch adds Space, a class representing a process's address space.
- Each Process has a Space.
- The Space owns the PageDirectory and all Regions in the Process.
This allows us to reorganize sys$execve() so that it constructs and
populates a new Space fully before committing to it.
Previously, we would construct the new address space while still
running in the old one, and encountering an error meant we had to do
tedious and error-prone rollback.
Those problems are now gone, replaced by what's hopefully a set of much
smaller problems and missing cleanups. :^)
We now build the kernel with partial UBSAN support.
The following -fsanitize sub-options are enabled:
* nonnull-attribute
* bool
If the kernel detects UB at runtime, it will now print a debug message
with a stack trace. This is very cool! I'm leaving it on by default for
now, but we'll probably have to re-evaluate this as more options are
enabled and slowdown increases.
This adds support for FUTEX_WAKE_OP, FUTEX_WAIT_BITSET, FUTEX_WAKE_BITSET,
FUTEX_REQUEUE, and FUTEX_CMP_REQUEUE, as well well as global and private
futex and absolute/relative timeouts against the appropriate clock. This
also changes the implementation so that kernel resources are only used when
a thread is blocked on a futex.
Global futexes are implemented as offsets in VMObjects, so that different
processes can share a futex against the same VMObject despite potentially
being mapped at different virtual addresses.
All users of this mechanism have been switched to anonymous files and
passing file descriptors with sendfd()/recvfd().
Shbufs got us where we are today, but it's time we say good-bye to them
and welcome a much more idiomatic replacement. :^)
