Ports: Add Python 3.9

The current version of our Python port (3.6.0) is over four years old by
now and has (or had, I haven't actually tried it in a while) some
limitations - time for an upgrade! The latest Python release is 3.9.1,
so I used that version. It's a from-scratch port, no patches are taken
from the previous port to ensure the smallest possible amount of code is
patched. The BuildPython.sh script is useful so I kept it, with some
tweaks. I added a short document explaining each patch to ease judging
their underlying problem and necessity in the future.

Compared to the old Python port, this one does support both the time
module as well as threading (at least _thread) just fine. Importing
modules written in C (everything in /usr/local/lib/python3.9/lib-dynload)
currently asserts in Serenity's dynamic loader, which is unfortunate but
probably solvable. Possibly related to #4642. I didn't try building
Python statically, which might be one possibility to circumvent this
issue.

I also renamed the directory to just "python3", which is analogous to
the Python 3.x package most Linux distributions provide. That implicitly
means that we likely will not support multiple versions of the Python
port at any given time, but again, neither do many other systems by
default. Recent versions are usually backwards compatible anyway though,
so having the latest shouldn't be a problem.
On the other hand bumping the version should now be be as simple as
updating the variables in version.sh, given that no new patches are
required.

These core modules to currently not build - I chose to ignore that for
now rather than adding more patches to make them work somehow, which
means they're fully unavailable. This should probably be fixed in
Serenity itself.

    _ctypes, _decimal, _socket, mmap, resource, termios

These optional modules requiring 3rd-party dependencies do currently not
build (even with depends="ncurses openssl zlib"). Especially the absence
of a readline port makes the REPL a bit painful to use. :^)

    _bz2, _curses, _curses_panel, _dbm, _gdbm, _hashlib, _lzma, _sqlite3,
    _ssl, _tkinter, _uuid, nis, ossaudiodev, readline, spwd, zlib

I did some work on LibC and LibM beforehand to add at least stubs of
missing required functions, it still encounters an ASSERT_NOT_REACHED()
/ TODO() every now and then, notably frexp() (implementations of that
can be found online easily if you want to get that working right now).
But then again that's our fault and not this port's. :^)

Ports/python3/package.sh

@@ -0,0 +1,38 @@
+#!/bin/bash ../.port_include.sh
+
+source version.sh
+
+port=python3
+version="${PYTHON_VERSION}"
+workdir="Python-${version}"
+useconfigure="true"
+files="${PYTHON_ARCHIVE_URL} ${PYTHON_ARCHIVE}
+https://www.python.org/ftp/python/${version}/Python-${version}.tar.xz.asc Python-${version}.tar.xz.asc"
+auth_type="sig"
+auth_import_key="E3FF2839C048B25C084DEBE9B26995E310250568"
+auth_opts="Python-${version}.tar.xz.asc Python-${version}.tar.xz"
+
+# We could say depends="ncurses openssl zlib" here, but neither of the _curses, _ssl, and zlib modules
+# build at the moment even with those available, so it's pointless.
+
+# FIXME: the --build value is detected correctly by the configure script (via config.guess in the Python source root),
+# but still needs to be set explicitly when cross compiling. Figure out how to not hardcode this.
+BUILD="x86_64-pc-linux-gnu"
+
+# FIXME: --enable-optimizations results in lots of __gcov_* linker errors
+configopts="--build=${BUILD} --without-ensurepip ac_cv_file__dev_ptmx=no ac_cv_file__dev_ptc=no"
+
+export BLDSHARED="${CC} -shared"
+
+if [ -x "$(command -v python3)" ]; then
+    # Check if major and minor version of python3 are matching
+    if ! python3 -c "import sys; major, minor, _ = map(int, '${PYTHON_VERSION}'.split('.')); sys.exit(not (sys.version_info.major == major and sys.version_info.minor == minor))"; then
+        echo "Error: python3 version does not match needed version to build ${PYTHON_VERSION}" >&2
+        echo "Build this Python version on your host using Toolchain/BuildPython.sh or install it otherwise and try again." >&2
+        exit 1
+    fi
+else
+    echo "Error: python3 is not installed but is required to build ${PYTHON_VERSION}" >&2
+    echo "Build this Python version on your host using Toolchain/BuildPython.sh or install it otherwise and try again." >&2
+    exit 1
+fi

Ports/python3/patches/README.md

@@ -0,0 +1,39 @@
+# Patches for Python 3.9 on SerenityOS
+
+## `define-have-sigset-t.patch`
+
+Ensures `HAVE_SIGSET_T` is defined, as we *do* have `sigset_t` but it's not detected properly due to some related functions being missing.
+
+## `define-py-force-utf8-locale.patch`
+
+Enforce UTF-8 as encoding by defining `_Py_FORCE_UTF8_LOCALE`.
+
+## `disable-setrlimit.patch`
+
+Disables check for `RLIMIT_CORE` and subsequent `setrlimit()` call. Would be enabled otherwise as we *do* have `<sys/resource.h>` and therefore `HAVE_SYS_RESOURCE_H`.
+
+## `fix-autoconf.patch`
+
+As usual, make the `configure` script recognize Serenity.
+
+## `fix-hidden-symbol-referenced-by-dso.patch`
+
+Fix a weird build issue of `python` and other provided binaries by marking the `main()` functions `Py_EXPORTED_SYMBOL`.
+
+```text
+hidden symbol `main' in Programs/python.o is referenced by DSO
+```
+
+Not sure what the proper fix for this is, but it works fine.
+
+## `remove-setlocale-from-preconfig.patch`
+
+Our stub implementation of `setlocale()` always returns `nullptr`, which the interpreter considers critical enough to exit right away.
+
+## `tweak-unsupported-printf-format-specifiers.patch`
+
+Replace uses of `%.Ns` with `%s` as the former is not supported by our `printf` implementation yet and would result in empty strings. It uses `snprintf` already, so this is safe.
+
+## `use-rtld-lazy-for-dlopenflags.patch`
+
+We have `RTLD_NOW` defined but don't actually support it, so use the provided `RTLD_LAZY` fallback. Doesn't help the dynamic library module import assertion though.

Ports/python3/patches/define-have-sigset-t.patch

@@ -0,0 +1,12 @@
+--- Python-3.9.1/Modules/posixmodule.h	2021-01-17 20:56:14.590000000 +0100
++++ Python-3.9.1/Modules/posixmodule.h	2021-01-17 20:56:34.207894812 +0100
+@@ -19,7 +19,8 @@
+ #endif /* MS_WINDOWS */
+ 
+ #if defined(PYPTHREAD_SIGMASK) || defined(HAVE_SIGWAIT) || \
+-        defined(HAVE_SIGWAITINFO) || defined(HAVE_SIGTIMEDWAIT)
++        defined(HAVE_SIGWAITINFO) || defined(HAVE_SIGTIMEDWAIT) || \
++        defined(__serenity__)
+ # define HAVE_SIGSET_T
+ #endif
+ 

Ports/python3/patches/define-py-force-utf8-locale.patch

@@ -0,0 +1,11 @@
+--- Python-3.9.1/Include/pyport.h	2021-01-17 20:45:44.417000000 +0100
++++ Python-3.9.1/Include/pyport.h	2021-01-17 20:46:07.865663659 +0100
+@@ -838,7 +838,7 @@
+ #  error "Py_TRACE_REFS ABI is not compatible with release and debug ABI"
+ #endif
+ 
+-#if defined(__ANDROID__) || defined(__VXWORKS__)
++#if defined(__ANDROID__) || defined(__VXWORKS__) || defined(__serenity__)
+    /* Ignore the locale encoding: force UTF-8 */
+ #  define _Py_FORCE_UTF8_LOCALE
+ #endif

Ports/python3/patches/disable-setrlimit.patch

@@ -0,0 +1,11 @@
+--- Python-3.9.1/Modules/faulthandler.c	2021-01-17 20:45:32.878000000 +0100
++++ Python-3.9.1/Modules/faulthandler.c	2021-01-17 20:45:33.006210297 +0100
+@@ -993,7 +993,7 @@
+     SetErrorMode(mode | SEM_NOGPFAULTERRORBOX);
+ #endif
+ 
+-#ifdef HAVE_SYS_RESOURCE_H
++#if 0
+     struct rlimit rl;
+ 
+     /* Disable creation of core dump */

Ports/python3/patches/fix-autoconf.patch

@@ -0,0 +1,55 @@
+--- Python-3.9.1/config.sub	2021-01-17 20:15:56.796000000 +0100
++++ Python-3.9.1/config.sub	2021-01-17 20:21:04.324828217 +0100
+@@ -1485,6 +1485,8 @@
+ 	-oss*)
+ 		os=-sysv3
+ 		;;
++	-serenity*)
++		;;
+ 	-svr4*)
+ 		os=-sysv4
+ 		;;
+--- Python-3.9.1/configure.ac	2021-01-17 20:33:50.524295313 +0100
++++ Python-3.9.1/configure.ac	2021-01-17 20:34:24.631127320 +0100
+@@ -382,6 +382,9 @@
+        # a lot of different things including 'define_xopen_source'
+        # in the case statement below.
+ 	case "$host" in
++	*-*-serenity*)
++		ac_sys_system=Serenity
++		;;
+ 	*-*-linux-android*)
+ 		ac_sys_system=Linux-android
+ 		;;
+@@ -428,6 +431,9 @@
+ AC_SUBST(_PYTHON_HOST_PLATFORM)
+ if test "$cross_compiling" = yes; then
+ 	case "$host" in
++	*-*-serenity*)
++		_host_cpu=$host_cpu
++		;;
+ 	*-*-linux*)
+ 		case "$host_cpu" in
+ 		arm*)
+--- Python-3.9.1/configure	2021-01-17 20:35:39.813757019 +0100
++++ Python-3.9.1/configure	2021-01-17 20:36:00.538654942 +0100
+@@ -3292,6 +3292,9 @@
+        # a lot of different things including 'define_xopen_source'
+        # in the case statement below.
+ 	case "$host" in
++	*-*-serenity*)
++		ac_sys_system=Serenity
++		;;
+ 	*-*-linux-android*)
+ 		ac_sys_system=Linux-android
+ 		;;
+@@ -3339,6 +3342,9 @@
+ 
+ if test "$cross_compiling" = yes; then
+ 	case "$host" in
++	*-*-serenity*)
++		_host_cpu=$host_cpu
++		;;
+ 	*-*-linux*)
+ 		case "$host_cpu" in
+ 		arm*)

Ports/python3/patches/fix-hidden-symbol-referenced-by-dso.patch

@@ -0,0 +1,22 @@
+--- Python-3.9.1/Programs/python.c	2021-01-18 08:25:32.203494270 +0100
++++ Python-3.9.1/Programs/python.c	2021-01-18 08:25:49.711418585 +0100
+@@ -9,7 +9,7 @@
+     return Py_Main(argc, argv);
+ }
+ #else
+-int
++Py_EXPORTED_SYMBOL int
+ main(int argc, char **argv)
+ {
+     return Py_BytesMain(argc, argv);
+--- Python-3.9.1/Programs/_testembed.c	2021-01-18 08:22:35.085000000 +0100
++++ Python-3.9.1/Programs/_testembed.c	2021-01-18 08:23:16.036082910 +0100
+@@ -1711,7 +1711,7 @@
+     {NULL, NULL}
+ };
+ 
+-int main(int argc, char *argv[])
++Py_EXPORTED_SYMBOL int main(int argc, char *argv[])
+ {
+     if (argc > 1) {
+         for (struct TestCase *tc = TestCases; tc && tc->name; tc++) {

Ports/python3/patches/remove-setlocale-from-preconfig.patch

@@ -0,0 +1,30 @@
+--- Python-3.9.1/Python/preconfig.c	2021-01-17 21:03:08.698000000 +0100
++++ Python-3.9.1/Python/preconfig.c	2021-01-17 21:03:47.828031544 +0100
+@@ -790,16 +790,6 @@
+ 
+     preconfig_get_global_vars(config);
+ 
+-    /* Copy LC_CTYPE locale, since it's modified later */
+-    const char *loc = setlocale(LC_CTYPE, NULL);
+-    if (loc == NULL) {
+-        return _PyStatus_ERR("failed to LC_CTYPE locale");
+-    }
+-    char *init_ctype_locale = _PyMem_RawStrdup(loc);
+-    if (init_ctype_locale == NULL) {
+-        return _PyStatus_NO_MEMORY();
+-    }
+-
+     /* Save the config to be able to restore it if encodings change */
+     PyPreConfig save_config;
+ 
+@@ -899,10 +889,6 @@
+     status = _PyStatus_OK();
+ 
+ done:
+-    if (init_ctype_locale != NULL) {
+-        setlocale(LC_CTYPE, init_ctype_locale);
+-        PyMem_RawFree(init_ctype_locale);
+-    }
+     Py_UTF8Mode = init_utf8_mode ;
+ #ifdef MS_WINDOWS
+     Py_LegacyWindowsFSEncodingFlag = init_legacy_encoding;

Ports/python3/patches/tweak-unsupported-printf-format-specifiers.patch

@@ -0,0 +1,22 @@
+--- Python-3.9.1/Python/getversion.c	2021-01-18 08:31:52.780000000 +0100
++++ Python-3.9.1/Python/getversion.c	2021-01-18 08:32:14.176848948 +0100
+@@ -9,7 +9,7 @@
+ Py_GetVersion(void)
+ {
+     static char version[250];
+-    PyOS_snprintf(version, sizeof(version), "%.80s (%.80s) %.80s",
++    PyOS_snprintf(version, sizeof(version), "%s (%s) %s",
+                   PY_VERSION, Py_GetBuildInfo(), Py_GetCompiler());
+     return version;
+ }
+--- Python-3.9.1/Modules/getbuildinfo.c	2021-01-18 08:54:23.766207240 +0100
++++ Python-3.9.1/Modules/getbuildinfo.c	2021-01-18 08:54:09.757000000 +0100
+@@ -43,7 +43,7 @@
+     if (!(*gitid))
+         gitid = "default";
+     PyOS_snprintf(buildinfo, sizeof(buildinfo),
+-                  "%s%s%s, %.20s, %.9s", gitid, sep, revision,
++                  "%s%s%s, %s, %s", gitid, sep, revision,
+                   DATE, TIME);
+     return buildinfo;
+ }

Ports/python3/patches/use-rtld-lazy-for-dlopenflags.patch

@@ -0,0 +1,11 @@
+--- Python-3.9.1/Python/pystate.c	2021-01-18 18:33:06.021000000 +0100
++++ Python-3.9.1/Python/pystate.c	2021-01-18 18:33:50.274359610 +0100
+@@ -223,7 +223,7 @@
+ 
+     interp->eval_frame = _PyEval_EvalFrameDefault;
+ #ifdef HAVE_DLOPEN
+-#if HAVE_DECL_RTLD_NOW
++#if defined(HAVE_DECL_RTLD_NOW) && !defined(__serenity__)
+     interp->dlopenflags = RTLD_NOW;
+ #else
+     interp->dlopenflags = RTLD_LAZY;