soft/build_system/README.md

# Mageia build system & tools

***This is an attempt to summarize, document and organize the whole tools making
the buildsystem here at Mageia.***

## Big picture

TODO _show_ what are all the pieces of the build system,
how they interact, who interacts with these, and what are the basic actions
of each actor, and expected reactions.

See possible docs:

 * [BS details on Mandriva wiki](http://wiki.mandriva.com/en/Build_System_Details)
 * [Dev tasks, BS theory on Mandriva wiki](http://wiki.mandriva.com/en/Development/Tasks/Packaging/BuildSystem/Theory)
 * [Dev tasks, BS overview on Mandriva wiki](http://wiki.mandriva.com/en/Development/Tasks/Packaging/BuildSystem/Overview)
 * [Packagers HOWTO](https://wiki.mageia.org/en/Packagers_Howto_start)

Objects:

 * source code
 * spec file
 * source package
 * compiled package
 * packages repository
 * system image
 * ?

Roles:

 * distribution release manager
 * build system admin
 * packages repository admin
 * packager
 * system (scheduled/triggered tasks)

----
## Tools included

Packager tools:

 * mgarepo - the tool used to manage RPM packages in a subversion repository on Mageia.

Building & pkg management tools:

 * iurt, emi & ulri 
 * youri - the suite uploading and managing the RPM repositories after the builds

Tools to build final images:

 * bcd - for installer ISOs
 * draklive - for live ISOs
 * ?

Reporting tools:

 * web


TODO testing tools? other final builds tools? how/where/when are isolinux/*, dosutils/*, misc/* and install/* (.iso and .img included) built in distrib/{arch}?

Details follow.

----

### bcd

 * BCD a new tool to build Mandriva ISO.
 * License: GNU LGPL 2.1

 * works: manually (can be automated?)
 * for: distribution release manager
 * input: distribution repository + specific configurations
 * output: installable ISO image of the distribution (what about live images? other images?)

### draklive

 * draklive is...
 * License: GNU GPL 2+
 * Code is in svn://svn.mageia.org/svn/soft/{draklive,draklive-install}

 * works: manually or automated
 * for: distribution release manager
 * input: distribution repository + specific configuration
 * output:

### iurt

 * Descriptions taken from header docs:
   * iurt: build packages inside clean chroots
   * emi: upload packages in queue when all the mandatory architectures are done
   * ulri: 
     * sends packages to the build nodes for each architecture
     * collects built packages and logs
 * TODOs are in comments in the code, should be checked for current validity.
 * License: GNU GPL v2+

 * iurt docs:
   * [iurt HOWTO on Mandriva wiki](http://wiki.mandriva.com/en/Development/Howto/Iurt)
   * [iurt analysis on Mandriva wiki](http://wiki.mandriva.com/en/Development/Tasks/Packaging/IurtAnalysis)

 * works: manually
 * for: packager
 * input: source package
 * output: compiled packages

### youri

 * YOURI stands for "Youri Offers an Upload & Repository Infrastucture".
 * It aims to build tools making management of a coherent set of packages easier.
   Managing a package repository involves many tasks, such as keeping packages
   tree tidy, generating packages indexes, synchronising bug report system,
   running coherency checks, checking for available updates, etc...
   Instead of a gazillion project-specific scripts, we aim to provide a generic
   package-format independant framework, so as to build coherent and robust tools.

 * Components:
   * youri-check allows to check packages
   * youri-upload allows to upload packages

 * License: Perl License

 * works: automatically and manually?
 * for: buildsystem/repository admin?
 * input: ?
 * output: ?

### mgarepo

 * mgarepo is the tool used to manage RPM packages in a subversion repository
   on Mageia.
 * It is a fork of the repsys tool used by Mandriva.
 * It is used to create, tag releases, generate .src.rpm, generate changelog,
   and request new package releases for build. It mostly acts as a interface
   to svn(1) commands and small task scripts run that on the build system side
   over ssh(1).
 * For more information, see mgarepo(8)
 * The discussion on the development of mageiarepo takes place on the
   Mageia developers mailing list.
 * License: 
 * Howtos: [mgarepo](https://wiki.mageia.org/en/Mgarepo)

 * works: manually
 * for: packager
 * input: ?
 * output: ?

### web

 * hosts components to report various states in the build system
   (upload queue, uploaded packages and build results, etc.)
 * License:

 * works: auto
 * for: anyone willing to checkout status info about the whole system
 * input: various data sources in the system (TODO details)
 * output: HTML documents and parsable data structures (JSON, text)

----
## Code

Code is available in:

 - [Subversion](svn://svn.mageia.org/svn/soft/build_system),
 - [Web view](http://svnweb.mageia.org/soft/build_system).

---
## TODO & BUGS

 * please check out [our quite extensive Bugzilla](https://bugs.mageia.org/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&component=BuildSystem&product=Infrastructure)
   and use it for further bugs.
 * see and compare/update from:
   * [Dev tasks for BS improvement on Mandriva Wiki](http://wiki.mandriva.com/en/Development/Tasks/Packaging/BuildSystem/Improvements)
   * [Proposals for SoC 2012](https://wiki.mageia.org/en/SummerOfCode2012#Projects_Proposals)
   * 


----
1	rda	6445	# Mageia build system & tools
2
3			***This is an attempt to summarize, document and organize the whole tools making
4			the buildsystem here at Mageia.***
5
6			## Big picture
7
8			TODO _show_ what are all the pieces of the build system,
9			how they interact, who interacts with these, and what are the basic actions
10			of each actor, and expected reactions.
11
12	rda	6447	See possible docs:
13
14	rda	6489	* [BS details on Mandriva wiki](http://wiki.mandriva.com/en/Build_System_Details)
15			* [Dev tasks, BS theory on Mandriva wiki](http://wiki.mandriva.com/en/Development/Tasks/Packaging/BuildSystem/Theory)
16			* [Dev tasks, BS overview on Mandriva wiki](http://wiki.mandriva.com/en/Development/Tasks/Packaging/BuildSystem/Overview)
17			* [Packagers HOWTO](https://wiki.mageia.org/en/Packagers_Howto_start)
18
19	rda	6446	Objects:
20
21			* source code
22			* spec file
23			* source package
24			* compiled package
25			* packages repository
26			* system image
27			* ?
28
29			Roles:
30
31			* distribution release manager
32			* build system admin
33			* packages repository admin
34			* packager
35			* system (scheduled/triggered tasks)
36
37	rda	6445	----
38	rda	6489	## Tools included
39	rda	6445
40	rda	6489	Packager tools:
41	rda	6445
42	rda	6489	* mgarepo - the tool used to manage RPM packages in a subversion repository on Mageia.
43
44			Building & pkg management tools:
45
46			* iurt, emi & ulri
47			* youri - the suite uploading and managing the RPM repositories after the builds
48
49			Tools to build final images:
50
51			* bcd - for installer ISOs
52			* draklive - for live ISOs
53			* ?
54
55			Reporting tools:
56
57	rda	6445	* web
58
59
60	rda	6489	TODO testing tools? other final builds tools? how/where/when are isolinux/, dosutils/, misc/* and install/* (.iso and .img included) built in distrib/{arch}?
61
62			Details follow.
63
64			----
65
66	rda	6445	### bcd
67
68			* BCD a new tool to build Mandriva ISO.
69			* License: GNU LGPL 2.1
70
71			* works: manually (can be automated?)
72			* for: distribution release manager
73			* input: distribution repository + specific configurations
74			* output: installable ISO image of the distribution (what about live images? other images?)
75
76	rda	6446	### draklive
77
78			* draklive is...
79			* License: GNU GPL 2+
80	rda	6489	* Code is in svn://svn.mageia.org/svn/soft/{draklive,draklive-install}
81	rda	6446
82	rda	6447	* works: manually or automated
83	rda	6446	* for: distribution release manager
84			* input: distribution repository + specific configuration
85			* output:
86
87	rda	6445	### iurt
88
89	rda	6447	* Descriptions taken from header docs:
90	pterjan	6448	* iurt: build packages inside clean chroots
91	rda	6445	* emi: upload packages in queue when all the mandatory architectures are done
92	pterjan	6448	* ulri:
93			* sends packages to the build nodes for each architecture
94			* collects built packages and logs
95	rda	6445	* TODOs are in comments in the code, should be checked for current validity.
96			* License: GNU GPL v2+
97
98	rda	6447	* iurt docs:
99	rda	6489	* [iurt HOWTO on Mandriva wiki](http://wiki.mandriva.com/en/Development/Howto/Iurt)
100			* [iurt analysis on Mandriva wiki](http://wiki.mandriva.com/en/Development/Tasks/Packaging/IurtAnalysis)
101	rda	6447
102	rda	6445	* works: manually
103			* for: packager
104	pterjan	6448	* input: source package
105			* output: compiled packages
106	rda	6445
107			### youri
108
109			* YOURI stands for "Youri Offers an Upload & Repository Infrastucture".
110			* It aims to build tools making management of a coherent set of packages easier.
111			Managing a package repository involves many tasks, such as keeping packages
112			tree tidy, generating packages indexes, synchronising bug report system,
113			running coherency checks, checking for available updates, etc...
114			Instead of a gazillion project-specific scripts, we aim to provide a generic
115			package-format independant framework, so as to build coherent and robust tools.
116
117			* Components:
118			* youri-check allows to check packages
119			* youri-upload allows to upload packages
120
121			* License: Perl License
122
123			* works: automatically and manually?
124			* for: buildsystem/repository admin?
125			* input: ?
126			* output: ?
127
128			### mgarepo
129
130			* mgarepo is the tool used to manage RPM packages in a subversion repository
131			on Mageia.
132			* It is a fork of the repsys tool used by Mandriva.
133			* It is used to create, tag releases, generate .src.rpm, generate changelog,
134			and request new package releases for build. It mostly acts as a interface
135			to svn(1) commands and small task scripts run that on the build system side
136			over ssh(1).
137			* For more information, see mgarepo(8)
138			* The discussion on the development of mageiarepo takes place on the
139			Mageia developers mailing list.
140			* License:
141	rda	6489	* Howtos: [mgarepo](https://wiki.mageia.org/en/Mgarepo)
142	rda	6445
143			* works: manually
144			* for: packager
145			* input: ?
146			* output: ?
147
148			### web
149
150			* hosts components to report various states in the build system
151			(upload queue, uploaded packages and build results, etc.)
152			* License:
153
154			* works: auto
155			* for: anyone willing to checkout status info about the whole system
156			* input: various data sources in the system (TODO details)
157			* output: HTML documents and parsable data structures (JSON, text)
158
159			----
160			## Code
161
162			Code is available in:
163
164			- [Subversion](svn://svn.mageia.org/svn/soft/build_system),
165			- [Web view](http://svnweb.mageia.org/soft/build_system).
166
167			---
168			## TODO & BUGS
169
170			* please check out [our quite extensive Bugzilla](https://bugs.mageia.org/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&component=BuildSystem&product=Infrastructure)
171			and use it for further bugs.
172	rda	6447	* see and compare/update from:
173	rda	6489	* [Dev tasks for BS improvement on Mandriva Wiki](http://wiki.mandriva.com/en/Development/Tasks/Packaging/BuildSystem/Improvements)
174			* [Proposals for SoC 2012](https://wiki.mageia.org/en/SummerOfCode2012#Projects_Proposals)
175	rda	6447	*
176	rda	6445
177
178	pterjan	6448	----