soft/build_system/README.md

# Mageia build system & tools

***This is an attempt to summarize, document and organize the whole tools making
the buildsystem here at Mageia.***

## Big picture

TODO _show_ what are all the pieces of the build system,
how they interact, who interacts with these, and what are the basic actions
of each actor, and expected reactions.

See possible docs:

 * [BS details on Mandriva wiki](http://wiki.mandriva.com/en/Build_System_Details)
 * [Dev tasks, BS theory on Mandriva wiki](http://wiki.mandriva.com/en/Development/Tasks/Packaging/BuildSystem/Theory)
 * [Dev tasks, BS overview on Mandriva wiki](http://wiki.mandriva.com/en/Development/Tasks/Packaging/BuildSystem/Overview)
 * [Packagers HOWTO](https://wiki.mageia.org/en/Packagers_Howto_start)

Objects:

 * source code
 * spec file
 * source package
 * compiled package
 * packages repository
 * system image
 * ?

Roles:

 * distribution release manager
 * build system admin
 * packages repository admin
 * packager
 * system (scheduled/triggered tasks)

----
## Tools included

Packager tools:

 * mgarepo - the tool used to manage RPM packages in a subversion repository on Mageia.

Building & pkg management tools:

 * iurt, emi & ulri 
 * youri - the suite uploading and managing the RPM repositories after the builds

Tools to build final images:

 * bcd - for installer ISOs
 * draklive - for live ISOs
 * ?

Reporting tools:

 * web


TODO testing tools? other final builds tools? how/where/when are isolinux/*, dosutils/*, misc/* and install/* (.iso and .img included) built in distrib/{arch}?

Details follow.

----

### bcd

 * BCD a new tool to build Mandriva ISO.
 * License: GNU LGPL 2.1

 * works: manually (can be automated?)
 * for: distribution release manager
 * input: distribution repository + specific configurations
 * output: installable ISO image of the distribution (what about live images? other images?)

### draklive

 * draklive is...
 * License: GNU GPL 2+
 * Code is in svn://svn.mageia.org/svn/soft/{draklive,draklive-install}

 * works: manually or automated
 * for: distribution release manager
 * input: distribution repository + specific configuration
 * output:

### iurt

 * Descriptions taken from header docs:
   * iurt: build packages inside clean chroots
   * emi: upload packages in queue when all the mandatory architectures are done
   * ulri: 
     * sends packages to the build nodes for each architecture
     * collects built packages and logs
 * TODOs are in comments in the code, should be checked for current validity.
 * License: GNU GPL v2+

 * iurt docs:
   * [iurt HOWTO on Mandriva wiki](http://wiki.mandriva.com/en/Development/Howto/Iurt)
   * [iurt analysis on Mandriva wiki](http://wiki.mandriva.com/en/Development/Tasks/Packaging/IurtAnalysis)

 * works: manually
 * for: packager
 * input: source package
 * output: compiled packages

### youri

 * YOURI stands for "Youri Offers an Upload & Repository Infrastucture".
 * It aims to build tools making management of a coherent set of packages easier.
   Managing a package repository involves many tasks, such as keeping packages
   tree tidy, generating packages indexes, synchronising bug report system,
   running coherency checks, checking for available updates, etc...
   Instead of a gazillion project-specific scripts, we aim to provide a generic
   package-format independant framework, so as to build coherent and robust tools.

 * Components:
   * youri-check allows to check packages
   * youri-upload allows to upload packages

 * License: Perl License

 * works: automatically and manually?
 * for: buildsystem/repository admin?
 * input: ?
 * output: ?

### mgarepo

 * mgarepo is the tool used to manage RPM packages in a subversion repository
   on Mageia.
 * It is a fork of the repsys tool used by Mandriva.
 * It is used to create, tag releases, generate .src.rpm, generate changelog,
   and request new package releases for build. It mostly acts as a interface
   to svn(1) commands and small task scripts run that on the build system side
   over ssh(1).
 * For more information, see mgarepo(8)
 * The discussion on the development of mageiarepo takes place on the
   Mageia developers mailing list.
 * License: 
 * Howtos: [mgarepo](https://wiki.mageia.org/en/Mgarepo)

 * works: manually
 * for: packager
 * input: ?
 * output: ?

### web

 * hosts components to report various states in the build system
   (upload queue, uploaded packages and build results, etc.)
 * License:

 * works: auto
 * for: anyone willing to checkout status info about the whole system
 * input: various data sources in the system (TODO details)
 * output: HTML documents and parsable data structures (JSON, text)

----
## Code

Code is available in:

 - [Subversion](svn://svn.mageia.org/svn/soft/build_system),
 - [Web view](http://svnweb.mageia.org/soft/build_system).

---
## TODO & BUGS

 * please check out [our quite extensive Bugzilla](https://bugs.mageia.org/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&component=BuildSystem&product=Infrastructure)
   and use it for further bugs.
 * see and compare/update from:
   * [Dev tasks for BS improvement on Mandriva Wiki](http://wiki.mandriva.com/en/Development/Tasks/Packaging/BuildSystem/Improvements)
   * [Proposals for SoC 2012](https://wiki.mageia.org/en/SummerOfCode2012#Projects_Proposals)
   * 


----
1	# Mageia build system & tools
2
3	***This is an attempt to summarize, document and organize the whole tools making
4	the buildsystem here at Mageia.***
5
6	## Big picture
7
8	TODO _show_ what are all the pieces of the build system,
9	how they interact, who interacts with these, and what are the basic actions
10	of each actor, and expected reactions.
11
12	See possible docs:
13
14	* [BS details on Mandriva wiki](http://wiki.mandriva.com/en/Build_System_Details)
15	* [Dev tasks, BS theory on Mandriva wiki](http://wiki.mandriva.com/en/Development/Tasks/Packaging/BuildSystem/Theory)
16	* [Dev tasks, BS overview on Mandriva wiki](http://wiki.mandriva.com/en/Development/Tasks/Packaging/BuildSystem/Overview)
17	* [Packagers HOWTO](https://wiki.mageia.org/en/Packagers_Howto_start)
18
19	Objects:
20
21	* source code
22	* spec file
23	* source package
24	* compiled package
25	* packages repository
26	* system image
27	* ?
28
29	Roles:
30
31	* distribution release manager
32	* build system admin
33	* packages repository admin
34	* packager
35	* system (scheduled/triggered tasks)
36
37	----
38	## Tools included
39
40	Packager tools:
41
42	* mgarepo - the tool used to manage RPM packages in a subversion repository on Mageia.
43
44	Building & pkg management tools:
45
46	* iurt, emi & ulri
47	* youri - the suite uploading and managing the RPM repositories after the builds
48
49	Tools to build final images:
50
51	* bcd - for installer ISOs
52	* draklive - for live ISOs
53	* ?
54
55	Reporting tools:
56
57	* web
58
59
60	TODO testing tools? other final builds tools? how/where/when are isolinux/, dosutils/, misc/* and install/* (.iso and .img included) built in distrib/{arch}?
61
62	Details follow.
63
64	----
65
66	### bcd
67
68	* BCD a new tool to build Mandriva ISO.
69	* License: GNU LGPL 2.1
70
71	* works: manually (can be automated?)
72	* for: distribution release manager
73	* input: distribution repository + specific configurations
74	* output: installable ISO image of the distribution (what about live images? other images?)
75
76	### draklive
77
78	* draklive is...
79	* License: GNU GPL 2+
80	* Code is in svn://svn.mageia.org/svn/soft/{draklive,draklive-install}
81
82	* works: manually or automated
83	* for: distribution release manager
84	* input: distribution repository + specific configuration
85	* output:
86
87	### iurt
88
89	* Descriptions taken from header docs:
90	* iurt: build packages inside clean chroots
91	* emi: upload packages in queue when all the mandatory architectures are done
92	* ulri:
93	* sends packages to the build nodes for each architecture
94	* collects built packages and logs
95	* TODOs are in comments in the code, should be checked for current validity.
96	* License: GNU GPL v2+
97
98	* iurt docs:
99	* [iurt HOWTO on Mandriva wiki](http://wiki.mandriva.com/en/Development/Howto/Iurt)
100	* [iurt analysis on Mandriva wiki](http://wiki.mandriva.com/en/Development/Tasks/Packaging/IurtAnalysis)
101
102	* works: manually
103	* for: packager
104	* input: source package
105	* output: compiled packages
106
107	### youri
108
109	* YOURI stands for "Youri Offers an Upload & Repository Infrastucture".
110	* It aims to build tools making management of a coherent set of packages easier.
111	Managing a package repository involves many tasks, such as keeping packages
112	tree tidy, generating packages indexes, synchronising bug report system,
113	running coherency checks, checking for available updates, etc...
114	Instead of a gazillion project-specific scripts, we aim to provide a generic
115	package-format independant framework, so as to build coherent and robust tools.
116
117	* Components:
118	* youri-check allows to check packages
119	* youri-upload allows to upload packages
120
121	* License: Perl License
122
123	* works: automatically and manually?
124	* for: buildsystem/repository admin?
125	* input: ?
126	* output: ?
127
128	### mgarepo
129
130	* mgarepo is the tool used to manage RPM packages in a subversion repository
131	on Mageia.
132	* It is a fork of the repsys tool used by Mandriva.
133	* It is used to create, tag releases, generate .src.rpm, generate changelog,
134	and request new package releases for build. It mostly acts as a interface
135	to svn(1) commands and small task scripts run that on the build system side
136	over ssh(1).
137	* For more information, see mgarepo(8)
138	* The discussion on the development of mageiarepo takes place on the
139	Mageia developers mailing list.
140	* License:
141	* Howtos: [mgarepo](https://wiki.mageia.org/en/Mgarepo)
142
143	* works: manually
144	* for: packager
145	* input: ?
146	* output: ?
147
148	### web
149
150	* hosts components to report various states in the build system
151	(upload queue, uploaded packages and build results, etc.)
152	* License:
153
154	* works: auto
155	* for: anyone willing to checkout status info about the whole system
156	* input: various data sources in the system (TODO details)
157	* output: HTML documents and parsable data structures (JSON, text)
158
159	----
160	## Code
161
162	Code is available in:
163
164	- [Subversion](svn://svn.mageia.org/svn/soft/build_system),
165	- [Web view](http://svnweb.mageia.org/soft/build_system).
166
167	---
168	## TODO & BUGS
169
170	* please check out [our quite extensive Bugzilla](https://bugs.mageia.org/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&component=BuildSystem&product=Infrastructure)
171	and use it for further bugs.
172	* see and compare/update from:
173	* [Dev tasks for BS improvement on Mandriva Wiki](http://wiki.mandriva.com/en/Development/Tasks/Packaging/BuildSystem/Improvements)
174	* [Proposals for SoC 2012](https://wiki.mageia.org/en/SummerOfCode2012#Projects_Proposals)
175	*
176
177
178	----