A bunch of files describing the process of seamless configuration, building, installation, deinstallation and packaging for a given piece of third-party software on a FreeBSD system. It also helps to ensure that all the other software may be required in build time or run time is present on the host.
This approach heavily relies on use of
make(1)
macros, and builds the software from its
source code (well, most of the time).
Some of the possible reasons why one may want to learn more about ports:
(And arbitrary combination any of the above.)
All we need is...
make(1)
— included in base (soon to
be replaced with BSD Make though).A recent ports source, somewhere under /usr/ports/
— has to be installed or checked out via SVN:
# svn checkout http://svn.freebsd.org/ports/head /usr/ports
sh(1)
, C or any affected language or
framework.Ports Collection
or Ports Tree
: a
directory tree with categories and with ports in the categories.
${PORTSDIR}/${CATEGORY}/${PORT}
where
files Makefile distinfo pkg-descr pkg-plist
Some of them may be missing or there may be other additional files, but this is the most typical.
# Created by: Dmitry Sivachenko <demon@FreeBSD.org>
# $FreeBSD$
PORTNAME= cld
PORTVERSION= 0.1
CATEGORIES= devel
MASTER_SITES= ${MASTER_SITE_GOOGLE_CODE}
PROJECTHOST= chromium-compact-language-detector
DISTNAME= compact-language-detector-${PORTVERSION}
MAINTAINER= demon@FreeBSD.org
COMMENT= Chromium compact language detector library
GNU_CONFIGURE= yes
USE_LDCONFIG= yes
.include <bsd.port.mk>
SHA256 (compact-language-detector-0.1.tar.gz) = b1a3b430c0d39c8a8731d7291c30d896dd0f63c3fd093ec6ceb275bb760d1d5c
SIZE (compact-language-detector-0.1.tar.gz) = 2498722
A port from the CLD (Compact Language Detector) library embedded in
Google's Chromium browser. The library detects the language from
provided UTF8 text (plain text or HTML). It's implemented in C++,
with very basic Python bindings.
WWW: https://code.google.com/p/chromium-compact-language-detector/
include/cld/base/basictypes.h
include/cld/base/build_config.h
include/cld/base/port.h
include/cld/base/string_util.h
include/cld/compact_lang_det.h
include/cld/encodings/compact_lang_det/letterscript_enum.h
include/cld/encodings/compact_lang_det/win/cld_basictypes.h
include/cld/encodings/compact_lang_det/win/cld_utf8statetable.h
include/cld/encodings/proto/encodings.pb.h
include/cld/encodings/public/encodings.h
include/cld/ext_lang_enc.h
include/cld/lang_enc.h
include/cld/languages/proto/languages.pb.h
include/cld/languages/public/languages.h
lib/libcld.la
lib/libcld.so
lib/libcld.so.0
libdata/pkgconfig/cld.pc
@dirrm include/cld/languages/public
@dirrm include/cld/languages/proto
@dirrm include/cld/languages
@dirrm include/cld/encodings/public
@dirrm include/cld/encodings/proto
@dirrm include/cld/encodings/compact_lang_det/win
@dirrm include/cld/encodings/compact_lang_det
@dirrm include/cld/encodings
@dirrm include/cld/base
@dirrm include/cld
--- Makefile.in.orig 2012-05-23 14:35:00.000000000 +0400
+++ Makefile.in 2013-02-08 15:37:27.000000000 +0400
@@ -282,7 +282,7 @@
# autogen.sh and cleanrepo.sh are script for maintainance use. Not for distribution.
# dist_noinst_SCRIPTS = autogen.sh
-pkgconfigdir = $(libdir)/pkgconfig
+pkgconfigdir = ${exec_prefix}/libdata/pkgconfig
pkgconfig_DATA = cld.pc
basic_test_SOURCES = tests/basic_test.cc
basic_test_CXXFLAGS = -Wall -fPIC -Isrc/ -I../src -O2 -DCLD_WINDOWS
Ports maintain an implicit default behavior:
fetch
(the source tarball)
extract
(the source tarball)
patch
(the extracted sources)
configure
(the patched sources)
build
(the configured sources)
install
(the built program)
package
(the installed program, optional)
deinstall
(the installed program, optional)
Each of the "phases" correspond to the a
make(1)
target. Actually a group of
targets (simplified):
phase-depends
pre-
phase
pre-
phase-script
do-
phase
post-
phase
post-
phase-script
Targets are like points of entries during the process, executed sequentially.
Variables are also handled by
make(1)
. They are often used as configuration parameters
in the Makefile
s.
NO_MTREE= yes
MAKE_ENV+= SHELL=${SH} NO_LINT=YES
PREFIX?= ${LOCALBASE}
ARCH!= ${UNAME} -p
CFLAGS:= ${CFLAGS:C/ $//}
$ make -V ARCH
Package listings can only work with substitutions,
determined by the value of PLIST_SUB
in the associated
Makefiles.
bin/cppi
%%NLS%%share/locale/de/LC_MESSAGES/cppi.mo
%%NLS%%share/locale/eo/LC_MESSAGES/cppi.mo
%%NLS%%share/locale/fi/LC_MESSAGES/cppi.mo
%%NLS%%share/locale/fr/LC_MESSAGES/cppi.mo
%%NLS%%share/locale/gl/LC_MESSAGES/cppi.mo
%%NLS%%share/locale/hr/LC_MESSAGES/cppi.mo
%%NLS%%share/locale/it/LC_MESSAGES/cppi.mo
%%NLS%%share/locale/ja/LC_MESSAGES/cppi.mo
%%NLS%%share/locale/pl/LC_MESSAGES/cppi.mo
%%NLS%%share/locale/sr/LC_MESSAGES/cppi.mo
%%NLS%%share/locale/sv/LC_MESSAGES/cppi.mo
%%NLS%%share/locale/uk/LC_MESSAGES/cppi.mo
%%NLS%%share/locale/vi/LC_MESSAGES/cppi.mo
$ make -V PLIST_SUB
When make(1)
does not offer enough
control, it is possible to use direct (imperative)
sh(1)
commands. They are attached to targets or
hooks.
makepatch:
@${MKDIR} ${FILESDIR}
@(cd ${PATCH_WRKSRC}; \
for i in `find . -type f -name '*.orig'`; do \
ORG=$$i; \
NEW=$${i%.orig}; \
OUT=${FILESDIR}`${ECHO} $${NEW} | \
${SED} -e 's|/|__|g' \
-e 's|^\.__|/patch-|'`; \
${ECHO} ${DIFF} -ud $${ORG} $${NEW} '>' $${OUT}; \
${DIFF} -ud $${ORG} $${NEW} > $${OUT} || ${TRUE}; \
done \
)
Shell commands can be also attached to certain events,
i.e. installation and deinstall of the port, in the
pkg-plist
file.
@exec /bin/ln -sf %D/bin/haddock-ghc-%%GHC_VERSION%% %D/bin/haddock || return true
@unexec /bin/rm -f %D/bin/haddock || return true
Makefile
s can have conditional sections,
controlled by variables or result of Boolean expressions.
.if defined(IA32_BINARY_PORT) && ${ARCH} != "i386"
.if ${ARCH} == "amd64" || ${ARCH} == "ia64"
.if !defined(HAVE_COMPAT_IA32_KERN)
IGNORE= requires a kernel with compiled-in IA32 compatibility
.elif !defined(HAVE_COMPAT_IA32_LIBS)
IGNORE= requires 32-bit libraries installed under /usr/lib32
.endif
_LDCONFIG_FLAGS=-32
LIB32DIR= lib32
.else
IGNORE= requires i386 (or compatible) platform to run
.endif
.else
LIB32DIR= lib
.endif
Loops may be used to walk whitespace-separated values stored in variables.
.for cat in ${CATEGORIES}
. if empty(VALID_CATEGORIES:M${cat})
@${ECHO_MSG} "${PKGNAME}: Makefile error: category ${cat} not in list of valid categories."; \
${FALSE};
. endif
.endfor
Makefiles may include contents of other Makefiles,
often with .mk
extension.
.if exists(${MASTERDIR}/Makefile.${ARCH}-${OPSYS})
.include "${MASTERDIR}/Makefile.${ARCH}-${OPSYS}"
USE_SUBMAKE= yes
.elif exists(${MASTERDIR}/Makefile.${OPSYS})
.include "${MASTERDIR}/Makefile.${OPSYS}"
USE_SUBMAKE= yes
.elif exists(${MASTERDIR}/Makefile.${ARCH})
.include "${MASTERDIR}/Makefile.${ARCH}"
USE_SUBMAKE= yes
.endif
.include <bsd.port.mk>
Makefiles may contain comments to help navigation in the sources.
# DO NOT COMMIT CHANGES TO THIS FILE BY YOURSELF, EVEN IF YOU DID NOT GET
# A RESPONSE FROM THE MAINTAINER(S) WITHIN A REASONABLE TIMEFRAME! ALL
# UNAUTHORISED CHANGES WILL BE UNCONDITIONALLY REVERTED!
Note that pkg-plist
uses a different
format for comments.
@comment $FreeBSD: head/devel/icmake/pkg-plist 308217 2012-12-04 09:13:30Z pgj $
PORTNAME= git
PORTVERSION= 1.8.2
PORTREVISION= 0
PORTEPOCH= 0
CATEGORIES= devel
MASTER_SITES= ${MASTER_SITE_GOOGLE_CODE}
PROJECTHOST= git-core
DISTFILES= ${PORTNAME}-${PORTVERSION}${EXTRACT_SUFX}
MAINTAINER= ports@FreeBSD.org
COMMENT= Distributed source code management tool
.include <bsd.port.mk>
# make fetch
===> git-1.8.2 depends on file: /usr/local/sbin/pkg - found
=> git-1.8.2.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch http://git-core.googlecode.com/files/git-1.8.2.tar.gz
git-1.8.2.tar.gz 100% of 4287 kB 420 kBps 00m00s
===> Fetching all distfiles required by git-1.8.2 for building
# make makesum
$ make
===> Building for git-1.8.2
"Makefile", line 348: Need an operator
"Makefile", line 388: Need an operator
"Makefile", line 406: Need an operator
"Makefile", line 443: Need an operator
"Makefile", line 603: Need an operator
"Makefile", line 605: Need an operator
"Makefile", line 606: Need an operator
"Makefile", line 607: warning: duplicate script for target "ifndef" ignored
"Makefile", line 608: Need an operator
"Makefile", line 609: Need an operator
"Makefile", line 610: warning: duplicate script for target "ifndef" ignored
"Makefile", line 611: Need an operator
"Makefile", line 613: Need an operator
"Makefile", line 614: Need an operator
...
USE_GMAKE= yes
MAKE_ARGS+= prefix="${PREFIX}"
Add a pkg-descr
file (mandatory) with a
longer, descriptive description of the ported application.
GIT is a "directory content manager" designed to handle absolutely massive
projects with speed and efficiency, and the release of the 2.6.12 (and later)
versions of the Linux kernel as well as more and more other projects switching
to it would indicate that it does this task well.
GIT falls in the category of distributed source code management tools, similar
to e.g. GNU Arch or Monotone (or, in the commercial world, BitKeeper). Every
GIT working directory is a full-fledged repository with full revision tracking
capabilities, not dependent on network access to a central server.
WWW: http://git-scm.com/
Note the WWW:
line — some tools or web
sites, e.g. FreshPorts may use
it.
First, the port should be built and installed with a
custom PREFIX
:
$ make PREFIX=/var/tmp/git
$ make install PREFIX=/var/tmp/git
Sometimes this can be skipped as the port does not
install too many files, e.g. audio/adplay
. In that
case, just add:
PLIST_FILES= bin/adplay %%DATADIR%%/adplug.db
PLIST_DIRS= %%DATADIR%%
$ make extract
===> git-1.8.2 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by git-1.8.2 for building
===> Extracting for git-1.8.2
=> SHA256 Checksum OK for git-1.8.2.tar.gz.
Find the places where the build or the installing process for the port may have gone wrong — and fix it!
Though, before a modifying a file, create a copy of it with
an .orig
extension. Once finished, use the
makepatch
target:
$ make makepatch
Once everything safely arrived to
/var/tmp/git
, use the utilities in the Ports Collection
to automatically generate a pkg-plist
file for
it:
$ /usr/ports/Tools/scripts/plist -Md -m `make -V MTREE_FILE` \
/var/tmp/`make -V PORTNAME` > pkg-plist
So far so good...
Testing ports in an isolated (chroot(8)
,
jail(8)
) environment is essential to find further
potential problems.
Both Ports Tinderbox
# cd /usr/ports/ports-mgmt/tinderbox
# make install
and Poudriere can be used for this.
# cd /usr/ports/ports-mgmt/poudriere
# make install
Create a jail for testing.
# poudriere jail -c -j 91amd64 -v 9.1-RELEASE -a amd64 -m ftp
Create a ports tree called local
. This
will contain the ports to test.
# poudriere ports -c -p local -F
But before we would proceed...
A fancy option for collecting custom ports is creating an overlay for the ports tree, that could be stored and published separately.
# cd /usr/ports/ports-mgmt/portshaker
# make install
Create the custom ports tree containing only the locally
created or modified ones. Add them to the configuration file of
portshaker
.
ports_trees="main local"
main_ports_tree="/usr/ports"
local_ports_tree="/usr/local/poudriere/ports/local/ports"
local_merge_from="ports local"
Create a file under
/usr/local/etc/portshaker.d
that describes the location of
the local ports tree:
#!/bin/sh
. /usr/local/share/portshaker/portshaker.subr
method="git"
git_clone_uri="git://github.com/ketchup/freebsd-ports.git"
run_portshaker_command $*
Use portshaker
to get the latest
merged ports tree with the modifications.
# portshaker -u ports -u local
# portshaker -v -m local
Use poudriere
to do a test build in an isolated
environment.
# poudriere testport -j 91amd64 -p local -o devel/git
origin: the containing directory for the port. Multiple packages may share origin.
Check the logs...
In file included from http.c:1:
http.h:6:23: warning: curl/curl.h: No such file or directory
http.h:7:23: warning: curl/easy.h: No such file or directory
In file included from http.c:1:
http.h:46: error: expected specifier-qualifier-list before 'CURLcode'
http.h:51: error: expected specifier-qualifier-list before 'CURL'
http.h:97: error: 'CURL_ERROR_SIZE' undeclared here (not in a function)
...
...
/usr/bin/perl Makefile.PL PREFIX='/prefix/git-1.8.2' INSTALL_BASE='' --localedir='/prefix/git-1.8.2/share/locale'
gmake[1]: /usr/bin/perl: Command not found
gmake[1]: *** [perl.mak] Error 127
gmake: *** [perl/perl.mak] Error 2
*** [do-build] Error code 1
Obviously applications may rely on other applications for building and running.
MAKE_ENV+= CURLDIR=${LOCALBASE}
BUILD_DEPENDS+= curl:${PORTSDIR}/ftp/curl
RUN_DEPENDS+= curl:${PORTSDIR}/ftp/curl
LIB_DEPENDS+= expat.6:${PORTSDIR}/textproc/expat2
USE_PERL5= yes
MAKE_ENV+= PERL_PATH=${PERL}
BUILD_DEPENDS+= p5-Error>=0:${PORTSDIR}/lang/p5-Error
RUN_DEPENDS+= p5-Error>=0:${PORTSDIR}/lang/p5-Error \
p5-Net-SMTP-SSL>=0:${PORTSDIR}/mail/p5-Net-SMTP-SSL
USE_PYTHON= yes
PLIST_SUB+= PYTHON_VER=${PYTHON_VER} PYTHON=""
CONFIGURE_ARGS+= --with-python=${LOCALBASE}/bin/python
Let us make the dependency on curl
and
expat
optional and introduce a user-configurable
option for it.
OPTIONS_DEFINE= CURL
OPTION_DEFAULT= # intentionally left blank = not a default option
CURL_DESC= Use curl
.include <bsd.port.options.mk>
.if ${PORT_OPTIONS:MCURL}
BUILD_DEPENDS+= curl:${PORTSDIR}/ftp/curl
RUN_DEPENDS+= curl:${PORTSDIR}/ftp/curl
LIB_DEPENDS+= expat.6:${PORTSDIR}/textproc/expat2
PLIST_SUB+= CURL=""
.else
MAKE_ENV+= NO_CURL=1 \
NO_EXPAT=1
PLIST_SUB+= CURL="@comment "
.endif
Various combinations of options could be set and
tested separately with poudriere
.
# poudriere options -c -j 91amd64 -p local devel/git
It easy to tell that the application expects presence of certains users on the system. If they are not present, the ports framework will create it automatically. (But it will not remove...)
USERS= git_daemon
GROUPS= git_daemon
USE_RC_SUBR= git_daemon
files/git_daemon.in
#! /bin/sh
# PROVIDE: git_daemon
# REQUIRE: DAEMON
# KEYWORD: shutdown
. /etc/rc.subr
name="git_daemon"
rcvar="git_daemon_enable"
load_rc_config $name
: ${git_daemon_user:=git_daemon}
: ${git_daemon_group:=git_daemon}
: ${git_daemon_enable:=NO}
: ${git_daemon_directory:=%%PREFIX%%/git}
: ${git_daemon_flags:=--syslog --reuseaddr --detach}
command="%%PREFIX%%/libexec/git-core/git-daemon"
command_args="${git_daemon_directory}"
PATH="${PATH}:%%PREFIX%%/libexec/git-core"
run_rc_command "$1"
Some of the files require us to copy them manually to the destination.
DISTFILES+= ${PORTNAME}-manpages-${PORTVERSION}${EXTRACT_SUFX}
EXTRACT_ONLY= ${PORTNAME}-${PORTVERSION}${EXTRACT_SUFX} \
${PORTNAME}-manpages-${PORTVERSION}${EXTRACT_SUFX}
post-install:
(cd ${WRKDIR}/man1/ && ${COPYTREE_SHARE} \* ${MANPREFIX}/man/man1)
(cd ${WRKDIR}/man5/ && ${COPYTREE_SHARE} \* ${MANPREFIX}/man/man5)
(cd ${WRKDIR}/man7/ && ${COPYTREE_SHARE} \* ${MANPREFIX}/man/man7)
# make makesum
Automated installation and removel of manual pages can be
invoked by using MAN1
, MAN5
, MAN7
variables in the Makefile
.
MAN5= gitattributes.5 \
githooks.5 \
gitignore.5 \
gitmodules.5 \
gitrepository-layout.5 \
gitweb.conf.5
MAN7= gitcli.7 \
gitcredentials.7 \
gitglossary.7 \
gittutorial.7 \
gitcore-tutorial.7 \
gitdiffcore.7 \
gittutorial-2.7 \
gitworkflows.7 \
gitrevisions.7 \
gitnamespaces.7
# MAN1 was left out intentionally due to its size.
Multiple packages may be created from a single port, and ports may share code via using slave ports.
PKGNAMESUFFIX= -curl
COMMENT= Distributed source code management tool with curl enabled
MASTERDIR= ${.CURDIR}/../git
DESCR= ${MASTERDIR}/pkg-descr
WITH_CURL= yes
CONFLICTS?= git-[0-9]*
DISTINFO_FILE= ${MASTERDIR}/distinfo
.include "${MASTERDIR}/Makefile"
Use of ports may be restricted based on certain circumstances.
CONFLICTS= git-curl-[0-9]*
BROKEN= does not build
IGNORE= not supported
RESTRICTED= not allowed to redistribute
Restricted ports must be added to the
LEGAL
file (in the root of the ports tree).
Sometimes ports get obsoleted, or they have to be renamed.
misc/git|misc/gnuit|2009-03-14|Renamed
Note: portshaker
supports this.
For changes visible to the consumers, it is usually a good
idea to add an entry to the UPDATING
(in the root of the
Ports Collection).
20121112:
AFFECTS: Users of devel/git
AUTHOR: wxs@FreeBSD.org
The git-daemon(1) process now runs as the git_daemon user. Please make
sure that this user has appropriate permissions to the repositories.
Note: portshaker
supports this.
For poudriere
, this is called a bulk
mode, where tests can be omitted for speeding up the process.
# poudriere bulk -j 91amd64 -p local devel/git
Example (may not work properly with Chrome)
The resulting packages (the port with all of its
dependencies) can be then put online and used with
pkg
:
PACKAGESITE: http://example.org/pkgng/freebsd:9:x86:64/
repos:
default : http://example.org/pkgng/
repo1 : http://somewhere.org/pkgng/repo1/
repo2 : http://somewhere.org/pkgng/repo2/
Submitting new ports or updates via the web site is easy.
Questions? Comments?
#bsdports
on IRC.Enjoy BSDCan 2013!