www.pudn.com > hipl.1.0.1.rar > HACKING

ABOUT
=====

This file contains developer information on policies in the HIPL
project.

TODO
====

- xx


AUTOCONF AND AUTOMAKE
=====================

Autoconf and automake tools help developers to avoid from "include
header dependency hell" so they are also used by HIPL in "tools" and
"test" directories. Standard linux kernel supports only plain old
Makefiles, so "linux" subdirectory is not controlled with
autoconf/automake. The "doc" directory uses plain old Makefile because
it uses the same documentation building system as "linux". Libinet6
has been imported from the Usagi project and uses only automake (no
autoconf).

If you need to add or remove source/header files or modify build
rules, edit "Makefile.am". If you need to modify dependencies, edit
"configure.ac". After editing the files, run "autoreconf". Autoreconf
may then update some files (config.h.in, config.sub, configure,
Makefile, Makefile.in) and you need to commit them too into the CVS
along with your new source and header files.

If you need to share a file between userspace and kernelspace, place
it in the kernelspace and modify userspace Makefile.am to create a
link to it from userspace (CVS does not accept links). For example,
"tools/Makefile.am" makes a local link to
"hipl/linux/net/ipv6/builder.c" by the following rule:

	builder.c:
        	$(LN_S) ../linux/net/ipv6/hip/builder.c .

CODING CONVENTIONS
==================

- Don't invent your own coding style. It will give the code an
  ununified look. Especially, if you don't follow the memory deallocation
  procedure below, you might even cause some memory leaks.

- All kernelspace code should follow the standard linux coding conventions
  listed in /Documentation/CodingStyle

- All code shared between userspace and kernel should follow kernel
  coding style.

- HIPL code should use the prefix "hip_" in all function and global variable
  definitions to avoid namespace pollution, especially in the
  libraries. The prefix is not necessary in userspace code, but the same
  naming conventions might be clearer to use anyway.

- The width of one line should be limited to 80 characters in
  userspace code. This rule may be relaxed in the kernel code because
  the space is very limited due to the 8 character indentation.

- All functions should return an error value instead of "ok" value. That, is
  zero for success and non-zero for failure.

- Use the HIP_IFE, HIP_IFEL macros to reduce the number of lines of code.

- Do not make unreadable code by nesting code too much:

  if (foo) {
     if (bar) {
        if(hello)
        {
           confuse_me();
        }
     }
  }

  Instead, join the conditional expressions (foo && bar && hello) or
  use goto (goto skip;).

- Declare variables right after beginning brace, not in the middle of the code:

  int foo {
    int i;  /* integers declared in the beginning of the function */
    char c;

    bar();

    for (i = 0; i < 100; i++) {
       int x; /* This is fine because it is after the beginning
                 brace of loop */
       hello(i, x);
    }
  }

  hipsock shares code with the rest of the code and it requires ISO C90
  style C code. Declare variables right after braces!

- Memory (de)allocation procedure is as follows:
 
	int f() {
		char *mem = NULL;
		HIP_IFEL(!(mem = HIP_ALLOC(256, 0)), -1, "alloc\n");

	  out_err:
		if (mem)
			free(mem);
		return err;
	}

QUICK-AND-DIRTY ARCH INSTRUCTIONS FOR DEVELOPERS
================================================

These instructions assume that you want to access the HIPL source code
using ssh. Note: tla and bazaar provide the same functionality but
bazaar was faster and more user friendly when this was written.

# Register an account:
#
baz register-archive hipl-dev@freelists.org--hipl \
       sftp://mkomu@hipl.hiit.fi/var/archive/hipl
baz my-default-archive hipl-dev@freelists.org--hipl

# Get a working copy from the main branch:
#
baz get hipl--main--2.6 hipl--main--2.6

# Copy the tla hook if you want to get your commit logs to the mailing list:
#
cp hipl--main--2.6/test/tla-hook ~/.arch-params/hook

# Different ways to commit code:
#
a) baz commit # Open's your default editor and commits.
b) emacs `tla make-log`; tla commit
c) baz commit -s "Explanation jaada jaada"

# How to check what your have modified in the tree
#
baz changes

# How to make a branch from the current version of the main branch
#
baz my-default archive # Make sure that you are accessing the HIPL archive
baz archive-setup hipl--mybranch--2.6
baz tag hipl--main--2.6 hipl--mybranch--2.6

# Get the new branch
#
baz get hipl--mybranch--2.6 hipl--mybranch--2.6

# How to synchronize branches... remember to make clean commits, i.e. do
# NOT commit a merge and local modifications in the same commit!
#
cd hipl--mybranch--2.6
baz changes # Check that the tree is unmodified
baz star-merge hipl--main--2.6

--- Optional ---

Below are some instructions on using revision libraries. They speed-up
tla/baz operations.

# Make a revision library (otherwise e.g. commit and changes take up to
# 15 mins!)
#
mkdir ~/.tla-rev-lib
baz my-revision-library ~/.tla-rev-lib/
baz library-config --greedy --sparse ~/.tla-rev-lib/

# Get a working copy from the main branch using revision libraries:
#
baz get --link hipl--main--2.6 hipl--main--2.6

Also, the tla usage is optimized using hard-linked revision
libraries.

# Run the command below in your working copy (e.g. monthly) if you
# experience a serious slow-down in the commit speed or you want to enable
# revision libraries in your local copy:
#
cd hipl--main--2.6
baz changes --link


ARCH POLICY
===========

*** Important: see that your umask is 0002 before you make any commits ***

hipl--main--2.6 is the "stable" branch for HIPL code. Do not put any
experimental code in there because other people are relying on that
the code works! Commit only minor bugfixes directly to the stable
branch, and use/create other branches to develop experimental
code. When you have tested and are sure that code works, you can merge
it into the stable branch.

Some points in the branch quality assurance:

- the code must compile as it is without errors
- compiler warning messages must be taken seriously and must not be ignored
  unless you really know what the message means. And we REALLY mean really.
- the base exchange should be manually tested between two hosts
- UPDATE packet exchange should work
- the code should not leak memory or do bad locking
- (all existing unit tests should be run without any errors (see TESTING))

DEBUGGING
=========

Debugging the kernel with gdb is not possible unless you're running
User Mode Linux and in such cases you may prefer manual debugging
statements in the code. It may be also your personal favourite to
prefer debugging statements over gdb in the userspace. HIPL provides a
set of wrappers for adding debugging statements in a concise way:

- HIP_DIE(arguments as for printk)
- HIP_ERROR(arguments as for printk)
- HIP_INFO(arguments as for printk) 
- HIP_DEBUG(arguments as for printk) 

The wrappers exists for many reasons. First, it is very convinient to
have the same syntax for the debug statements indepently of whether
you degugging in kernel or userspace.

Second, an "ad hoc" debug statement mechanism would make it more
difficult to adopt HIPL into production environment because it would
be awkward to comment all the superfluous debug messages. The
superfluous debug messages can be suppressed by a single switch to
configure (CONFIG_HIP_DEBUG) if an unified interface for debugging is
used.

Third reason for the existance of wrappers is that a uniform
importancy level of debug messages can be enforced. For example, error
messages are more important than info messages and info messages are
more important than debug messages. One benefit of the importancy of
levels is that the debug messages can be cathegorized into two groups:
the ones that are always included in a build and the ones that are
included only in a development build. HIP_DEBUG belongs to the latter
group and the other functions belong to the second group.

The fourth reason for justifying the wrappers is that some programs
cannot be run interactively on the screen in production
environment. For example, the HIP daemon is preferred to be run as a
background process in production environment by the end-users. The
debug statements cannot be printed on the screen in such a case;
instead, the statements have to passed via syslog. On the other hand,
the developer wants to run the daemon interactively. The wrappers make
it possible to please both of the groups easily.

Debug statements should not be removed from code because someone may
have use for the debug statement later on. The preferred way is to put a
"_" in front of the debug function name to prevent the debug statement
to ever to be shown (indepently of whether the build is a development
or production build):

- _HIP_DIE(..)
- _HIP_ERROR(..)
- _HIP_INFO(.. 
- _HIP_DEBUG(..) 

DOCUMENTATION
=============

Doxygen is now used for documenting HIPL. Manual for how to use doxygen
documentation, is found from here:
http://www.stack.nl/~dimitri/doxygen/manual.html

To create doxygen documentation in HTML format, execute "make doxygen". Doxygen will create documentation under doc/doxy/html.

Simple example, how to use some doxygen features:

/** Description of a #define. */
#define HIP_ENABLED 1

/**
 * Description of a global variable.
 * This can continue also to multiple lines,
 * just like previous #define description.
 */
int hip_sock = -1;

/**
 * This is a function description. Parameter documentation is in
 * form "@param  description". Return value is documented
 * using "@return description". Todos are documented using
 * "/*! \todo description */". Then the todos are listed in doxygen
 * created documentation as list.
 *
 * @param name This is a function parameter description.
 * @return This is a return value description.
 */
int test(char *name)
{
	/*! \todo
	 * Make contents for this function!!!
	 */
}


SUBMITTING PATCHES
==================

HIPL is still work in progress. The code is being constantly developed
and it may be difficult to merge your contribution to the HIPL code.

If you still want to submit patches to the HIPL project, see that the
patch follows the CVS POLICY (see above). Build your patches against
the HEAD branch in CVS.  It would be nice if you would build one (or
multiple) unit test case to validate your patch.

TESTING
=======

The project has a tailored unit testing environment which makes it
possible to run tests in the HIP kernel module and userspace. It is
recommended that the developer builds a new test case when creating
new functionality or repairing old functionality. Still, there are
some tangled functionality, especially in the kernel, that is
very difficult to test with unit testing. Manual black box testing
should be used to cover changes in the code in such cases.

CREATING A TARBALL OF THE SOURCE CODE
=====================================

There is a "make dist" target which builds a tarball of the hipl source   
code. It includes only the files which are necessary. The included files  
are specified in SOURCES and HEADER rules in */Makefile.am, but it is also
possible to force some extra files in top most Makefile.am.


PREBUILT BINARY PACKAGES
========================

Redhat
------

Note well: Currently the status of the packaging scripts is in very initial
phase. They are not tested throughly.

Two separate packages are planned to be provided by HIPL: one for user
space software and the other for kernel and kernel modules. Currently only
scripts for building user space packages exist.


There are scripts in directory test/packaging that can be used to create
binary packages. The aim for these packages is for easier testing of HIPL
software. With the scripts, RPM (e.g. for Fedora Core) and DEB (for Debian)
packages can be created. 

To create the RPM package, type: "./autogen && ./configure && make rpm".
This will create a source package (in tar.gz form in the same
directory) which will be used when building the RPM file. After the
script has run successfully, follow the instructions given by the
script. That is, login as root and copy the source package (e.g.
hipl-userspace-1.0-1.tgz) to directory /usr/src/redhat/SOURCES. Then
issue the command "rpmbuild -ba
/test/packaging/hiplx.spec".  When everyting
is finished successfully, there will be a binary RPM package
"/usr/src/rpm/RPMS/i386/hipl-userspace-1.0-1.i386.rpm". The package
can then be installed with rpm -i.

For more info about rpm packaging, please see:

http://www.redhat.com/docs/books/max-rpm/max-rpm-html/index.html
http://fedoranews.org/tchung/htmldoc/htmldoc.spec

Debian
------

DEB package is created similarly: "./autogen && ./configure && make
deb".
The result is both binary and source debian packages:
- Binary
Binary Debian package can be found in test/packaging/ directory
with the following name: hipl-1.0-1_i386.deb.
The package can be installed using dpkg, e.g. "dpkg -i
hipl-1.0-1_i386.deb".
- Source
Debian source package is made of three files which are:

	hipl-1.0.diff.gz
	hipl-1.0.dsc
	hipl-1.0.orig.tar.gz

and can be found under directory test/packaging/hipl-1.0-deb-src/


HOW TO CREATE KERNEL PACKAGES
=============================

Debian/Ubuntu
-------------

To create the package from vanilla sources for Debian distribution
or any distributions based on Debian (e.g. Ubuntu Dapper Drake),
follow the following instructions.

* Download and extract the vanilla kernel source (I got it from
(ftp://ftp.funet.fi/pub/linux/kernel/v2.6)
I extract it to /usr/src and I added my username to src group
by typing adduser my_username src
* cd linux-2.6.xx
* Patch the kernel with the hip patches you can find under hipl/patches directory
* cp /boot/config-something .config

In order to build the binary package type the following:
* fakeroot make-kpkg --initrd --append-to-version=.hipl --revision=custom.1.0 kernel_image

Instead, if you would like to produces  a  debianised  package  of  the  Linux kernel sources type the following:
* fakeroot make-kpkg --initrd --append-to-version=.hipl --revision=custom.1.0 kernel_source

After the kernel has been successfully compiled, you will find the
debian packages in the top directory.

To install it type:
dpkg -i kernel-image-2.6.16.5.hipl_1.0_i386.deb
and reboot. The debian package should already include the new 
installed kernel for grub.
If you need grub check this:
http://newbiedoc.sourceforge.net/system/kernel-pkg.html#GRUB-KERNEL-PKG

Main reference (.)
http://newbiedoc.sourceforge.net/system/kernel-pkg.html


Redhat notes
------------

The instructions below did not work anymore. Instead, I followed these
instructions:

http://grid-it.cnaf.infn.it/index.php?rpmbuild&type=1
see: test/packaging/kernel-2.6.17.14.spec 

To create the package from vanilla sources (preferred method) for
Fedora, follow the instructions of this document:

http://www.openvps.org/Plone/docs/developer/kernelrpm
http://www.fedoraforum.org/forum/archive/index.php/t-28290.html

In short:
* download and extract kernel sources
* cd linux-2.6.xx
* apply hip patches
* cp /boot/config-something .config
* make rpm # or bin-rpm??

To install, do this:
* rpm -i /usr/src/redhat/RPMS/i386/kernel-2.6.xx.rpm
* /sbin/new-kernel-pkg --mkinitrd --depmod --install --make-default 2.6.xx

NEW-KERNEL-PKG IS NECESSARY AS THE RPM DOES NOT INCLUDE INITRD AND
THE KERNEL IS NOT INCLUDED IN GRUB CONFIGURATION.

That will create and install vanilla rpm. However, if you want to use
the Fedora/Redhat patches to improve e.g. hardware support, use the
instructions below.

To create rpm for Fedora patched kernel, you need to download the
kernel from somewhere from here:

  http://download.fedora.redhat.com/pub/fedora/linux/core/updates/

For RH/Enterprise, you need to have an account on rhn.redhat.com to
get the latest packages. Browse/and search the channels list.

I basically followed these instructions:

  http://tqmcube.com/fc3_custom_kernel.htm

This is what I did:

  * rpm -ivh kernel-2.6.10-1.760_FC3.src.rpm
  * I modified /usr/src/redhat/SPECS/kernel-2.6.spec to include the HIP
    patches:

     %define rhbsys .beet

     # HIPL patches
     Patch190: simple-beet-ph-patch-v1.0-2.6.16.5
     Patch191: interfamily-beet-ph-patch-v1.0-2.6.16.5
     Patch192: policy-sleep-2.6.16.5-v2.patch
     Patch193: hipmod-2.6.16.5-v1.patch 
     # HIP patches
     %patch190 -p1
     %patch191 -p1
     %patch192 -p1
     %patch193 -p1

   * copied the patches to /usr/src/redhat/SOURCES
   * rpmbuild --target i686 -ba /usr/src/redhat/SPECS/kernel-2.6.16.9.spec

UPGRADING THE KERNEL - DEPRACATED
=================================

# note: there is a perl script called "hashline" diff which is more
# convinient to use in upgrading the kernel (to check for added/deleted
# files)

% 
% cd hipl--fix--2.6/linux
% patch -p1 <~/down/patch-2.6.7 2>&1| tee /tmp/PATCH_LOG
% grep ^patching /tmp/PATCH_LOG|cut -d' ' -f3 >/tmp/CHANGES
% for i in `cat /tmp/CHANGES`; do tla id $i 2>&1; done >/tmp/TLA_LOG
% tla add `perl -ne 'print "$_" if ($n); $n=0; $n=1 if
    (/id: untagged file/)' /tmp/LINUX_DIRS
% for i in `cat /tmp/LINUX_DIRS`; do tla id $i 2>&1; done
    >/tmp/TLA_DIR_LOG
% tla add `perl -ne 'print "$_" if ($n); $n=0; $n=1 if
    (/id: untagged file/)'