setrlimit/getrlimit: fix prlimit64 syscall use for 32-bit CPUs

Commit 95e38b37 ("add support for systems without legacy setrlimit/getrlimit
syscalls") has added use of the prlimit64 syscall in getrlimit and setrlimit
functions. This change causes memory corruption on getrlimit call for 32-bit
CPUs like ARC, as ARC doesn't have ugetrlimit syscall and uses prlimit64.
Also, setrlimit has been broken by prlimit64 call on 32-bit CPUs like, i386,
ARM, ARC.

For the prlimit64 syscall the kernel expects an rlimit struct with 64-bit fields,
but on 32-bit CPUs without _FILE_OFFSET_BITS=64 the struct rlimit has 32-bit
fields.

Add safe implementations of getrlimit, setrlimit, prlimit for 32-bit CPUs with a
local struct rlimit64 variable for use in the prlimit64 syscall.
For 64-bit CPUs and configurations with _FILE_OFFSET_BITS=64 use
getrlimit, setrlimit, prlimit as aliases to getrlimit64, setrlimit64 and
prlimit64. Add a new function prlimit64.

Tested on aarch64, arm, i386, arc.

Fixes: 95e38b37 ("add support for systems without legacy setrlimit/getrlimit syscalls")
Signed-off-by: Pavel Kozlov <pavel.kozlov@synopsys.com>

1 files changed, 36 insertions, 0 deletions

diff --git a/libc/sysdeps/linux/common/prlimit64.c b/libc/sysdeps/linux/common/prlimit64.c
new file mode 100644
index 000000000..6f57b939e
--- /dev/null
+++ b/libc/sysdeps/linux/common/prlimit64.c
@@ -0,0 +1,36 @@
+/*  Copyright (C) 2023 uClibc-ng
+ *  An prlimit64() - get/set resource limits Linux specific syscall.
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, see
+ *  <http://www.gnu.org/licenses/>.
+ */
+
+#include <sys/resource.h>
+#include <sysdep.h>
+
+#if defined(__NR_prlimit64)
+
+int
+prlimit64 (__pid_t pid, enum __rlimit_resource resource,
+	   const struct rlimit64 *new_rlimit, struct rlimit64 *old_rlimit)
+{
+	return INLINE_SYSCALL (prlimit64, 4, pid, resource, new_rlimit,
+			       old_rlimit);
+}
+
+# if __WORDSIZE == 64 || defined (__USE_FILE_OFFSET64)
+strong_alias_untyped(prlimit64, prlimit)
+# endif
+
+#endif
\ No newline at end of file