oracle / solaris-userland Goto Github PK

Open Source software in Solaris using gmake based build system to drive building various software components.

License: Other

Makefile 23.25% Roff 12.31% DTrace 0.08% C 35.87% Shell 3.04% HTML 0.36% Perl 2.81% M4 0.05% Python 1.53% CSS 0.02% JavaScript 18.97% Clean 0.08% XSLT 0.05% C++ 1.38% Ruby 0.01% Tcl 0.01% Java 0.01% DIGITAL Command Language 0.02% Assembly 0.15% sed 0.02%

solaris

solaris-userland's Introduction

Getting started with the Userland Consolidation

This repository contains the build recipes, local modifications, and IPS package manifests for most of the free and open source software that is packaged for Oracle Solaris 11.4 and published in the official Oracle Solaris package repository on pkg.oracle.com.

Some open source software in Solaris is maintained and packaged outside of the Userland Consolidation. Source code for those components may be available from https://www.oracle.com/downloads/opensource/solaris-source-code-downloads.html.

Getting Started

This README provides a very brief overview of the gate (i.e., source code repository), how to retrieve a copy, and how to build it. Detailed documentation about the Userland gate can be found in the doc directory.

Overview

The Userland consolidation maintains a project at

 https://github.com/oracle/solaris-userland

That repo contains build recipes, patches, IPS (i.e., pkg(7)) manifests, and other files necessary to download, prep, build, test, package and publish open source software. The build infrastructure makes use of hierarchical Makefiles which provide dependency and recipe information for building the components. In order to build the contents of the Userland gate, you need to clone it. Since you are reading this, you may already have.

Getting the Bits

The canonical repository internal to Oracle is stored in Mercurial, and is mirrored to an external Git repository on GitHub. In order to build or develop in the gate, you will need to clone it. For the external Git repository you can do so with the following command:

$ git clone https://github.com/oracle/solaris-userland /scratch/clone

This will create a replica of the various pieces that are checked into the source code management system, but it does not retrieve the community source archives associated with the gate content. To download the community source associated with your cloned workspace, you will need to execute the following:

$ cd /scratch/clone/components
$ gmake download

This will use GNU make and the downloading tool in the gate to walk through all of the component directories downloading and validating the community source archives from the gate machine or their canonical source repository.

There are two variation to this that you may find interesting. First, you can cause gmake(1) to perform its work in parallel by adding -j (jobs) to the command line. Second, if you are only interested in working on a particular component, you can change directories to that component's directory and use gmake download from that to only get its source archive.

Building the Bits.

You can build individual components or the contents of the entire gate.

Component build

If you are only working on a single component, you can just build it using following:

Setup the workspace for building components

$ cd (your-workspace)/components ; gmake setup

Build the individual component

$ cd (component-dir) ; gmake publish

Complete Top Down build

Complete top down builds are also possible by simply running $ tools/full-build # see --help for options

That is generally wrapper around $ cd (your-workspace) $ # cleanup your workspace to pristine state $ gmake publish $ # examine the log files and provide a summary

Help

See https://www.oracle.com/solaris/technologies/free-open-source-software.html for information on support for free and open source software provided for Oracle Solaris.

Discussion forums are available at:

Contributing

Instead of submitting a pull request, please follow our contribution guide.

Security

Please consult the security guide for our security vulnerability reporting and disclosure process.

License

Most of the content of this repo is released under the Common Development and Distribution License (CDDL) as shown at https://oss.oracle.com/licenses/.

Modifications to upstream projects are generally licensed under the same terms as the upstream project unless otherwise specified in the files providing the changes.

solaris-userland's People

Contributors

Stargazers

Watchers

solaris-userland's Issues

looking for Inf/NaN vs inf/nan patches for SPARC Solaris 11 gcc 5

At present gcc 5.4.0 shipped by Oracle has a non-standard way of printing Inf and NaN on SPARC Solaris 11.
(see the code below). Could someone please point out to use for relevant patches (ideally for gcc 7, but the one for gcc 5 would do, too). For some reason they never made it upstream, as far as I can tell.
Thanks!

/* inftest.c */
#include <stdio.h>
int main(void)
{
    double val = 10;
    for(int i = 0; i < 10000; ++i) {
        val = val * val;
    }
    printf("%g\n", val);
    printf("%g\n", val-val);
}
/* compile and run (with Oracle's gcc 5.4.0)
  $ gcc inftest.c
  $ ./a.out
  Inf
  NaN
*/
/* with '-std=c99' or '-std=c1x' it works correctly!
  $ gcc -std=c1x inftest.c
  $ ./a.out
  inf
  nan

With vanilla gcc 7.2 there is no difference, it's always Inf/NaN.
*/

gcc13 installation does not install static libstdc++.a

While the gcc13 build produces both

prototype/sparc/usr/gcc/13/lib/sparcv9/libstdc++.a
prototype/sparc/usr/gcc/13/lib/libstdc++.a

the manifest does not install these files.

When attempting to build rust this missing library results in an “Empty search path given via '-L’” error.

kstat2 -p kstat:/vm/usage/memory:mem_zfs do not work on Solaris zone

related change:
29402471 Add support for the parallel build of individual components.

the command in shared-macros.mk

ZFS_MEMORY_PAGES := $(shell $(KSTAT2) -p kstat:/vm/usage/memory:mem_zfs | cut -f 2)

do not work in the Solaris zone, causing confusing script errors

Upstream userland patches

Hello,

I am the maintainer of https://github.com/cloudbase/cloudbase-init .

Would it be possible to upstream the patches from https://github.com/oracle/solaris-userland/tree/master/components/cloudbase-init to the main cloudbase-init repo?

The files will be commited as-is or with slight code modifications (after passing through the review process), using the existent License headers Apache License and Copyright Copyright (c) 2020, Oracle and/or its affiliates.

The community would greatly appreciate this contribution.

Thank you,
Adrian Vladu

Typo in solaris-userland/components/net-snmp/Makefile, change 'B' with 'D'

MIB_MOBULES.sparc
should read
MIB_MODULES.sparc

MIB_MOBULES += $(MIB_MODULES.$(MACH))
should read
MIB_MODULES += $(MIB_MODULES.$(MACH))

racecondition()

File: /components/x11/app/gfx-utils/sun-src/fbconfig/fbconfig.c#L633

sprintf(*config_prog_path, xorg_prog_fmt, vis_ident_name);
		if (access(*config_prog_path, X_OK) == 0) {
			return (0);	/* Have a device-specific program */
}

I believe this indicates a security flaw, If an attacker can change anything along the path between the call access() and the files actually used, attacker may exploit the race condition.

Request team to have a look and validate.

rust 1.53.0 can't bootstrap rust 1.61.0

The previous rust version distributed in the Solaris Userland git repository was 1.53.0, on 26 July 2021.
The current rust version in Userland is 1.61.0 distributed 26 May 2022.

Version 1.53.0 is not capable of compiling 1.61.0, and there doesn't appear to have been a binary package update that provides version 1.61.0, or any version that is capable of compiling 1.61.0.

Version 1.53.0 is also not new enough to compile the current version of Firefox that is being distributed in Userland.

Is there a plan to address this?

building rust 1.60.0 on SPARC with 1.59.0 fails w/ lint error

It's probably really an upstream problem, but could be solved by patching the rust distribution (removing a test case) so that it doesn't fail and abort the build when testing the lint-docs component due to inline assembly not being supported on SPARC archictectures:

Building stage0 tool lint-docs (sparcv9-sun-solaris)
    Finished release [optimized] target(s) in 1.55s
warning: the code example in lint `named_asm_labels` in /scratch/clone/components/rust/rustc/rustc-1.60.0-src/compiler/rustc_lint/src/builtin.rs failed to generate the expected output: did not find lint `named_asm_labels` in output of example, got:

error[E0472]: inline assembly is unsupported on this target
 --> lint_example.rs:5:9
  |
5 |         asm!("foo: bar");
  |         ^^^^^^^^^^^^^^^^


error: aborting due to previous error


Rustbook (sparcv9-sun-solaris) - rustc

The easy workaround is to build with -i make option and ignore the failure.

setup for solaris-userland fails because host ipkg.us.oracle.com doesn't resolve

If you follow the instructions in the README.md, after git cloning the solaris-userland repository

user@hemlock:/scratch/clone/components$ gmake setup
Generating component list...
Generating component dependencies...
building tools...
/usr/gnu/bin/make -C ../tools clean
make[1]: Entering directory '/scratch/clone/tools'
make[1]: Leaving directory '/scratch/clone/tools'
/usr/gnu/bin/make -C ../tools setup
make[1]: Entering directory '/scratch/clone/tools'
make[1]: Leaving directory '/scratch/clone/tools'
Generating pkglint(1) cache from CANONICAL_REPO http://ipkg.us.oracle.com/solaris11/trunk...
pkglint(1) failed, please see /scratch/clone/sparc/logs/naughty-canonical-repo-actions
gmake: *** [Makefile:158: /scratch/clone/sparc/pkglint-cache] Error 2
user@hemlock:/scratch/clone/components$

and looking at the log file:

Lint engine setup...
Error: Unable to create reference image:   Framework error: code: E_COULDNT_RESOLVE_HOST (6) reason: Could not resolve host: ipkg.us.oracle.com
URL: 'http://ipkg.us.oracle.com/solaris11/trunk/versions/0/'

pkglint: Unable to create reference image:   Framework error: code: E_COULDNT_RESOLVE_HOST (6) reason: Could not resolve host: ipkg.us.oracle.com
URL: 'http://ipkg.us.oracle.com/solaris11/trunk/versions/0/'


real        1.8
user        1.3
sys         0.2
TARGET_EXIT_CODE 2

Presumably either the ipkg.us.oracle.com host should be resolvable (and reachable, and serving that content) or there should be instructions on what to change to remove this.

ModuleNotFoundError: No module named 'pkg'

When building packages I get this error. I have tried a few packages with the same result?
Is it me or has something changed in latest version of userland or is it a problem in latest Solaris SRU.
i.e.

cd /scratch/solaris-userland/components/gtar
/usr/bin/touch /scratch/solaris-userland/components/gtar/build/amd64/.installed
abort: there is no Mercurial repository here (.hg not found)
/usr/bin/pkgfmt -c -fv2 gtar.p5m
/usr/bin/pkgmogrify -D PERL_ARCH=i86pc-solaris-thread-multi-64 -D PERL_VERSION=5.22 -D IPS_COMPONENT_RE_VERSION=1\\.32 -D COMPONENT_RE_VERSION=1\\.32 -D PYTHON_2.7_ONLY=# -D PYTHON_3.4_ONLY=# -D PYTHON_3.5_ONLY=# -D PYTHON_3.7_ONLY=# -D SQ=\' -D DQ=\" -D Q=\" -I/scratch/solaris-userland/components/gtar -D SOLARIS_11_3_ONLY="#" -D SOLARIS_11_5_ONLY="#" -D SOLARIS_11_3_4_ONLY="" -D SOLARIS_11_4_5_ONLY="" -D SOLARIS_11_4_ONLY="" -D PY3_ABI3_NAMING="#" -D PY3_CPYTHON_NAMING="" -D ARC_CASE="" -D TPNO="53607" -D BUILD_VERSION="5.11-11.5.0.0.0.66.0" -D OS_RELEASE="5.11" -D SOLARIS_VERSION="2.11" -D PKG_SOLARIS_VERSION="11.5" -D CONSOLIDATION="userland" -D CONSOLIDATION_CHANGESET="" -D CONSOLIDATION_REPOSITORY_URL="https://github.com/oracle/solaris-userland.git" -D COMPONENT_VERSION="1.32" -D IPS_COMPONENT_VERSION="1.32" -D HUMAN_VERSION="" -D COMPONENT_ARCHIVE_URL="https://ftp.gnu.org/gnu/tar/tar-1.32.tar.bz2" -D COMPONENT_PROJECT_URL="https://www.gnu.org/software/tar" -D COMPONENT_NAME="tar" -D HG_REPO="" -D HG_REV="" -D HG_URL="" -D GIT_COMMIT_ID="" -D GIT_REPO="" -D GIT_TAG="" -D MACH="i386" -D MACH32="i86" -D MACH64="amd64" -D PUBLISHER="nightly" -D PUBLISHER_LOCALIZABLE="userland-localizable" -D BRANCHID="11.5.0.0.0.66.0" gtar.p5m \
        /scratch/solaris-userland/transforms/license-changes /scratch/solaris-userland/transforms/variant-cleanup /scratch/solaris-userland/transforms/autopyc /scratch/solaris-userland/transforms/defaults /scratch/solaris-userland/transforms/actuators /scratch/solaris-userland/transforms/depends /scratch/solaris-userland/transforms/devel /scratch/solaris-userland/transforms/docs /scratch/solaris-userland/transforms/locale /scratch/solaris-userland/transforms/standard-python-libraries /scratch/solaris-userland/transforms/python-rename /scratch/solaris-userland/transforms/python-3-groups /scratch/solaris-userland/transforms/python-3-soabi /scratch/solaris-userland/transforms/python-3-no-32bit /scratch/solaris-userland/transforms/ruby-tests /scratch/solaris-userland/transforms/puppet /scratch/solaris-userland/transforms/libtool-drop /scratch/solaris-userland/transforms/incorporate /scratch/solaris-userland/transforms/publish-cleanup /scratch/solaris-userland/transforms/legacy /scratch/solaris-userland/transforms/auto_fmri | \
        sed -e '/^$/d' -e '/^#.*$/d' | uniq >/scratch/solaris-userland/components/gtar/build/manifest-i386-gtar.mogrified
/scratch/solaris-userland/tools/userland-mangler -D /scratch/solaris-userland/components/gtar/build/prototype/i386/mangled -d /scratch/solaris-userland/components/gtar/build/prototype/i386/mangled -d /scratch/solaris-userland/components/gtar/build/prototype/i386 -d /scratch/solaris-userland/components/gtar/build -d /scratch/solaris-userland/components/gtar -d tar-1.32 -d /scratch/solaris-userland/licenses -m /scratch/solaris-userland/components/gtar/build/manifest-i386-gtar.mogrified >/scratch/solaris-userland/components/gtar/build/manifest-i386-gtar.mangled
Traceback (most recent call last):
  File "/scratch/solaris-userland/tools/userland-mangler", line 38, in <module>
    import pkg.fmri
ModuleNotFoundError: No module named 'pkg'
gmake: *** [/scratch/solaris-userland/make-rules/ips.mk:462: /scratch/solaris-userland/components/gtar/build/manifest-i386-gtar.mangled] Error 1

Thanks,

Andrew

"gmake download" fails to correctly locate various original source distributions

When one is outside the Oracle network and therefore the INTERNAL_ARCHIVE_MIRROR, http://userland.us.oracle.com/source-archives is not available... various downloads fail because the source files are no longer available from the hardcoded locations in the Makefiles. For example in ksh93 it will attempt to do:

/scratch/clone/tools/userland-fetch --file INIT.2012-08-01.tgz --url 'http://www2.research.att.com/~astopen/download/tgz/INIT.2012-08-01.tgz' --hash 'sha256:c40cf57e9b2186271a9c362a560aa4a6e25ba911a8258ab931d2bbdbce44cfe5'

however INIT.2012-08-01.tgz is no longer available at that URL.

It would be helpful if all the external component source locations were revalidated and updated in the Makefiles.

Samba: Component "samba-tool" is missing

The component "samba-tool" seems to be missing in the userland build of Solaris 11.4 SRU 37 (and previous SRUs).

The man page is there, though:

'''
root@pollux:~# pkg contents -m samba | grep samba-tool
file 1c44ec3ca6cb72661f9806111141721be2644042 chash=1a38b689fabb5291e363a0221155d40e22f26d5e facet.doc.man=all group=bin mode=0444 owner=root path=usr/share/man/man8/samba-tool.8 pkg.content-hash=file:sha512t_256:a6be1c235b1151be7d6c0433358f27237b93edeb36b8a810c4153c763529534c pkg.content-hash=gzip:sha512t_256:2ee27870359c05ea9e2d1b6d2fe059fa6c0ff7e6b927d002ee587929e805714b pkg.csize=7361 pkg.size=25922
'''

Maybe, the missing tool has just to be added to /components/samba/samba.p5m.

mailman having uid 98 triggers compliance check for system accounts

Not directly an issue but a cross package and registration/reservation thing:
The manifest nails mailman to uid 98 while system ids (less than 100) are checked by a compliance report, which then yields an alarm as it checks against an inline list of 'known' account names:
Command: /usr/lib/compliance/tests/OSC-25505-ids-reserved-uids-sce.sh
Yield: Invalid User with Reserved UID: mailman

Noticed on usr/lib/compliance/tests/OSC-25505-ids-reserved-uids-sce.sh from pkg:/security/compliance/[email protected] and pkg://solaris/mail/[email protected],5.11-0.175.3.23.0.4.0:20170802T150919Z)
)
I don't know about 'system account' registration policies, especially with regard to foss packages, but at least wanted to ring the bell ;-)

Best regards
Jens

Clone and Download not working

Cloned the Git Repo.
Get a lot of errors during "game download"

hergen@solaris11:/Source$ git clone https://github.com/oracle/solaris-userland /export/home/hergen/Source/Solaris-userland
Cloning into '/export/home/hergen/Source/Solaris-userland'...
remote: Enumerating objects: 118293, done.
remote: Counting objects: 100% (7177/7177), done.
remote: Compressing objects: 100% (2985/2985), done.
remote: Total 118293 (delta 4304), reused 6572 (delta 3991), pack-reused 111116
Receiving objects: 100% (118293/118293), 160.55 MiB | 4.37 MiB/s, done.
Resolving deltas: 100% (54099/54099), done.
Updating files: 100% (9256/9256), done.
hergen@solaris11:/Source$ cd Solaris-userland/components/
hergen@solaris11:/Source/Solaris-userland/components$ gmake download
/bin/bash: /usr/perl5/5.32/bin/perl: No such file or directory
/bin/mkdir -p /export/home/hergen/Source/Solaris-userland/i386
Generating component list...
Generating component dependencies...
/bin/bash: /usr/perl5/5.32/bin/perl: No such file or directory
Generating component list...
Generating component dependencies...
/bin/bash: /usr/perl5/5.32/bin/perl: No such file or directory
/bin/mkdir -p /export/home/hergen/Source/Solaris-userland/i386/logs
download a2ps
make[1]: Entering directory '/export/home/hergen/Source/Solaris-userland/components/a2ps'
/bin/bash: /usr/perl5/5.32/bin/perl: No such file or directory
/bin/bash: /usr/perl5/5.32/bin/perl: No such file or directory
/usr/php/7.3/bin/php-config: No such file or directory
/usr/php/8.0/bin/php-config: No such file or directory
/export/home/hergen/Source/Solaris-userland/tools/userland-fetch --file a2ps-4.14.tar.gz --url 'https://ftp.gnu.org/gnu/a2ps/a2ps-4.14.tar.gz' --hash 'sha256:f3ae8d3d4564a41b6e2a21f237d2f2b104f48108591e8b83497500182a3ab3a4' --sigurl 'https://ftp.gnu.org/gnu/a2ps/a2ps-4.14.tar.gz.sig'
Source a2ps-4.14.tar.gz... not found, skipping file copy
Source http://userland.us.oracle.com/source-archives/a2ps-4.14.tar.gz...
downloading... failed
Can't open url http://userland.us.oracle.com/source-archives/a2ps-4.14.tar.gz
<urlopen error [Errno 8] node name or service name not known>
Source https://ftp.gnu.org/gnu/a2ps/a2ps-4.14.tar.gz...
downloading...
validating signature... ok
validating hash... ok
/usr/bin/touch a2ps-4.14.tar.gz
make[1]: Leaving directory '/export/home/hergen/Source/Solaris-userland/components/a2ps'
download aalib
make[1]: Entering directory '/export/home/hergen/Source/Solaris-userland/components/aalib'
/bin/bash: /usr/perl5/5.32/bin/perl: No such file or directory
/bin/bash: /usr/perl5/5.32/bin/perl: No such file or directory
/usr/php/7.3/bin/php-config: No such file or directory
/usr/php/8.0/bin/php-config: No such file or directory
/export/home/hergen/Source/Solaris-userland/tools/userland-fetch --file aalib-1.4rc5.tar.gz --url 'http://downloads.sourceforge.net/aa-project/aalib-1.4rc5.tar.gz' --hash 'sha256:fbddda9230cf6ee2a4f5706b4b11e2190ae45f5eda1f0409dc4f99b35e0a70ee'
Source aalib-1.4rc5.tar.gz... not found, skipping file copy
Source http://userland.us.oracle.com/source-archives/aalib-1.4rc5.tar.gz...
downloading... failed
Can't open url http://userland.us.oracle.com/source-archives/aalib-1.4rc5.tar.gz
<urlopen error [Errno 8] node name or service name not known>
Source http://downloads.sourceforge.net/aa-project/aalib-1.4rc5.tar.gz...
downloading...
validating signature... skipped (no signature URL)
validating hash... ok
/usr/bin/touch aalib-1.4rc5.tar.gz
make[1]: Leaving directory '/export/home/hergen/Source/Solaris-userland/components/aalib'
download acpidump
make[1]: Entering directory '/export/home/hergen/Source/Solaris-userland/components/acpidump'
/bin/bash: /usr/perl5/5.32/bin/perl: No such file or directory
/bin/bash: /usr/perl5/5.32/bin/perl: No such file or directory
/usr/php/7.3/bin/php-config: No such file or directory
/usr/php/8.0/bin/php-config: No such file or directory
/export/home/hergen/Source/Solaris-userland/tools/userland-fetch --file pmtools-20071116.tar.gz --url 'https://01.org/sites/default/files/page/pmtools-20071116.tar.gz' --hash 'sha256:91751774976e39f6237efd0326eb35196a9346220b92ad35894a33283e872748'
Source pmtools-20071116.tar.gz... not found, skipping file copy
Source http://userland.us.oracle.com/source-archives/pmtools-20071116.tar.gz...
downloading... failed
Can't open url http://userland.us.oracle.com/source-archives/pmtools-20071116.tar.gz
<urlopen error [Errno 8] node name or service name not known>
Source https://01.org/sites/default/files/page/pmtools-20071116.tar.gz...
downloading... failed
Can't open url https://01.org/sites/default/files/page/pmtools-20071116.tar.gz
HTTP Error 404: Not Found
Cannot download the archives! Terminating...
make[1]: *** [/export/home/hergen/Source/Solaris-userland/make-rules/prep.mk:39: pmtools-20071116.tar.gz] Error 100
make[1]: Leaving directory '/export/home/hergen/Source/Solaris-userland/components/acpidump'
gmake: *** [Makefile:249: acpidump.nosetup] Error 2
hergen@solaris11:/Source/Solaris-userland/components$

Missing Firefox and Cargo

What happened to Firefox and Cargo? They were in components last week (1/4/20) and now (1/12/20) they're gone :-(