On Fri, Sep 03, 2004 at 10:48:33AM -0700, Jay Lan wrote:
> Support of project concept is certainly one thing we need to consider.
> We have received requests to support project in CSA.
> - jay
> >Can/will this support project based accounting ?
> >i.e. Users being member of several projects. Each project having
> >multiple users as members. And the user is allowed to decide which
> >project-id each container is going to run under / be accounted to ?
Here is an update of the Enhanced Accounting HOWTO.
Here is the ChangeLog:
6/09/2004: Guillaume Thouvenin
- Add a section concerning projects based accounting
proposed by Jan-Frode Myklebust.
- Add a section concerning the EAC (Enhanced Accounting
And here is the proposal:
Enhanced Linux accounting Howto?
According to discussion on the lse-tech mailing-list, here are some
stages that are required to improve accounting.
1) Improve accounting structure
The current BSD-accounting structure doesn't have enough informations.
Metrics computed by CSA module can be added to BSD accounting. According to
other discussion (like Andi Kleen's comment on the patch I wrote when I wanted
to add CSA IO values in the BSD accounting ( http://lkml.org/lkml/2004/8/2/70 )
the current method to get metrics about blocks/char read/write is not accurate
since most writes can be accounted to pdflsuhd.
2) Group of processes management
We need to be able to manage groups of processes as it's clear that a major
accounting improvement is the per-job accounting. I don't know if "job" is the
right noun. There are several implementation that already exist and some of them
are already in the kernel. The property needed here is that if a process is in
a container, its children will be in the same container. Different
implementations can be:
- PAGG + JOB (job)
- ELSA (bank)
- CKRM (class)
- CPUSET (a cpuset of all CPUs can act as a container)
The interface between kernel and user space application can be a new virtual
file system (like CKRM does with /rcfs) but it can also be a device driver with
ioctl operations (like ELSA does). Both solutions are interesting and need to
be discussed. We can notice that CPUSET is in 2.6.9-rc1-mm2 tree and PAGG has
been removed from -mm tree because there is a lack of code that uses PAGG.
3) Project based accounting
It could be interesting to add a support for project based accounting. It
means that a user can be a member of one or more projects and he is allowed
to decide which project-id each container is going to run. This could be done
inside the Enhanced Accounting Core module.
4) Data presentation
We can have several different implementations.
5) General overview
KERNEL SPACE | USER SPACE (or MODULE)
| BSD accounting | |
| + | | ------------
| CSA |<=======>| |
---------------- | | Enhanced |
| | Accounting |
------------- | | Core |
| group of |<=======>| |
| processes | | ------------
| manager | |
Communication between EAC (Enhanced Accounting Core) and kernel space can be
done via virtual fs or a device. The goal of EAC is to keep a trace of the
different groups of processes during the accounting period using the group of
processes management module. The idea is that the group of processes management
can send a message to EAC when a process ended to indicate the job
ID if any. The other thing that has to be done by EAC is the data presentation.
Here is an example of what can be done:
a) First we can add processes into containers using the GPM (group of
processes manager) module. For example we can add an ftp server with
pid #123 and a daemon ssh with pid #234. Thus, inside the GPM you have:
container 1 -> 123
container 2 -> 234
b) Now, a user can login via ssh, so, sshd will create new children. Thus,
inside the GPM you will have something like:
container 1 -> 123
container 2 -> 234 333 334 335
c) As soon as a process is terminated, the EAC must be aware of that fact
(using a signal from GPM for example). There is just this modification
that is needed. With the signal, the GPM will transfer information like
pid, jib (job ID), command, ... to the EAC. This will allow to keep trace
about what happens in the system. Thus, in the EAC we keep information
about all processes that are finished.
d) If the sysadmin asks information about process 234, the EAC will know that
it belongs to container 2 and there are other processes in this container.
As all processes are exited, accounting information are written in the
accounting file (like it is currently with BSD accounting). With those
informations (accounting + job information) we can do per-job accounting.
6) Enhanced Accounting Core (EAC) implementation
The EAC can be implemented in the user space or as a kernel module.
We will see later differences between both implementation. The EAC has
o It will manage a group of processes historic. It means that it
will keep a trace concerning processes that belong to the same
container. We don't care about how container are managed, we just
need to know when a new process is created and if he is in a
container. It means that the job manager (or group of processes
manager) must provide a mechanism to share this information. If
we decide to implement EAC in user space, information can be pass
through a signal or a device. If it's a module, we can add an hook
in the code of the job manager.
o It will provide routine to add/remove processes from container. It
will act as a wrapper for routine that already exist in the job
o It will get information from the accounting file. Probably from
BSD accounting extended with CSA information. Those informations
will be used in conjunction with the group of process historic to
provide the enhanced accounting which is a per-job accounting
o Finally, EAC we will support the project based accounting as describe
in section 3)
I am working on the EAC implementation to check how we can make it work.