Contents of /trunk/mkinitrd-magellan/busybox/docs/busybox.net/FAQ.html
Parent Directory | Revision Log
Revision 816 -
(show annotations)
(download)
(as text)
Fri Apr 24 18:33:46 2009 UTC (15 years, 1 month ago) by niro
File MIME type: text/html
File size: 55237 byte(s)
Fri Apr 24 18:33:46 2009 UTC (15 years, 1 month ago) by niro
File MIME type: text/html
File size: 55237 byte(s)
-updated to busybox-1.13.4
1 | <!--#include file="header.html" --> |
2 | |
3 | <h3>Frequently Asked Questions</h3> |
4 | |
5 | This is a collection of some of the more frequently asked questions |
6 | about BusyBox. Some of the questions even have answers. If you |
7 | have additions to this FAQ document, we would love to add them, |
8 | |
9 | <h2>General questions</h2> |
10 | <ol> |
11 | <li><a href="#getting_started">How can I get started using BusyBox?</a></li> |
12 | <li><a href="#configure">How do I configure busybox?</a></li> |
13 | <li><a href="#build">How do I build BusyBox with a cross-compiler?</a></li> |
14 | <li><a href="#build_system">How do I build a BusyBox-based system?</a></li> |
15 | <li><a href="#kernel">Which Linux kernel versions are supported?</a></li> |
16 | <li><a href="#arch">Which architectures does BusyBox run on?</a></li> |
17 | <li><a href="#libc">Which C libraries are supported?</a></li> |
18 | <li><a href="#commercial">Can I include BusyBox as part of the software on my device?</a></li> |
19 | <li><a href="#external">Where can I find other small utilities since busybox does not include the features I want?</a></li> |
20 | <li><a href="#demanding">I demand that you to add <favorite feature> right now! How come you don't answer all my questions on the mailing list instantly? I demand that you help me with all of my problems <em>Right Now</em>!</a></li> |
21 | <li><a href="#helpme">I need help with BusyBox! What should I do?</a></li> |
22 | <li><a href="#contracts">I need you to add <favorite feature>! Are the BusyBox developers willing to be paid in order to fix bugs or add in <favorite feature>? Are you willing to provide support contracts?</a></li> |
23 | </ol> |
24 | |
25 | <h2>Troubleshooting</h2> |
26 | <ol> |
27 | <li><a href="#bugs">I think I found a bug in BusyBox! What should I do?!</a></li> |
28 | <li><a href="#backporting">I'm using an ancient version from the dawn of time and something's broken. Can you backport fixes for free?</a></li> |
29 | <li><a href="#init">Busybox init isn't working!</a></li> |
30 | <li><a href="#sed">I can't configure busybox on my system.</a></li> |
31 | <li><a href="#job_control">Why do I keep getting "sh: can't access tty; job control turned off" errors? Why doesn't Control-C work within my shell?</a></li> |
32 | </ol> |
33 | |
34 | <h2>Misc. questions</h2> |
35 | <ol> |
36 | <li><a href="#tz">How do I change the time zone in busybox?</a></li> |
37 | </ol> |
38 | |
39 | <h2>Programming questions</h2> |
40 | <ol> |
41 | <li><a href="#goals">What are the goals of busybox?</a></li> |
42 | <li><a href="#design">What is the design of busybox?</a></li> |
43 | <li><a href="#source">How is the source code organized?</a> |
44 | <ul> |
45 | <li><a href="#source_applets">The applet directories.</a></li> |
46 | <li><a href="#source_libbb">The busybox shared library (libbb)</a></li> |
47 | </ul> |
48 | </li> |
49 | <li><a href="#optimize">I want to make busybox even smaller, how do I go about it?</a></li> |
50 | <li><a href="#adding">Adding an applet to busybox</a></li> |
51 | <li><a href="#standards">What standards does busybox adhere to?</a></li> |
52 | <li><a href="#portability">Portability.</a></li> |
53 | <li><a href="#tips">Tips and tricks.</a> |
54 | <ul> |
55 | <li><a href="#tips_encrypted_passwords">Encrypted Passwords</a></li> |
56 | <li><a href="#tips_vfork">Fork and vfork</a></li> |
57 | <li><a href="#tips_short_read">Short reads and writes</a></li> |
58 | <li><a href="#tips_memory">Memory used by relocatable code, PIC, and static linking.</a></li> |
59 | <li><a href="#tips_kernel_headers">Including Linux kernel headers.</a></li> |
60 | </ul> |
61 | </li> |
62 | <li><a href="#who">Who are the BusyBox developers?</a></li> |
63 | </ol> |
64 | |
65 | |
66 | <hr /> |
67 | <h1>General questions</h1> |
68 | |
69 | <hr /> |
70 | <h2><a name="getting_started">How can I get started using BusyBox?</a></h2> |
71 | |
72 | <p> If you just want to try out busybox without installing it, download the |
73 | tarball, extract it, run "make defconfig", and then run "make". |
74 | </p> |
75 | <p> |
76 | This will create a busybox binary with almost all features enabled. To try |
77 | out a busybox applet, type "./busybox [appletname] [options]", for |
78 | example "./busybox ls -l" or "./busybox cat LICENSE". Type "./busybox" |
79 | to see a command list, and "busybox appletname --help" to see a brief |
80 | usage message for a given applet. |
81 | </p> |
82 | <p> |
83 | BusyBox uses the name it was invoked under to determine which applet is |
84 | being invoked. (Try "mv busybox ls" and then "./ls -l".) Installing |
85 | busybox consists of creating symlinks (or hardlinks) to the busybox |
86 | binary for each applet in busybox, and making sure these links are in |
87 | the shell's command $PATH. The special applet name "busybox" (or with |
88 | any optional suffix, such as "busybox-static") uses the first argument |
89 | to determine which applet to run, as shown above. |
90 | </p> |
91 | <p> |
92 | BusyBox also has a feature called the |
93 | <a name="standalone_shell">"standalone shell"</a>, where the busybox |
94 | shell runs any built-in applets before checking the command path. This |
95 | feature is also enabled by "make allyesconfig", and to try it out run |
96 | the command line "PATH= ./busybox ash". This will blank your command path |
97 | and run busybox as your command shell, so the only commands it can find |
98 | (without an explicit path such as /bin/ls) are the built-in busybox ones. |
99 | This is another good way to see what's built into busybox. |
100 | Note that the standalone shell requires CONFIG_BUSYBOX_EXEC_PATH |
101 | to be set appropriately, depending on whether or not /proc/self/exe is |
102 | available or not. If you do not have /proc, then point that config option |
103 | to the location of your busybox binary, usually /bin/busybox. |
104 | (So if you set it to /proc/self/exe, and happen to be able to chroot into |
105 | your rootfs, you must mount /proc beforehand.) |
106 | </p> |
107 | <p> |
108 | A typical indication that you set CONFIG_BUSYBOX_EXEC_PATH to proc but |
109 | forgot to mount proc is: |
110 | <pre> |
111 | $ /bin/echo $PATH |
112 | /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11 |
113 | $ echo $PATH |
114 | /bin/sh: echo: not found |
115 | </pre> |
116 | |
117 | <hr /> |
118 | <h2><a name="configure">How do I configure busybox?</a></h2> |
119 | |
120 | <p> Busybox is configured similarly to the linux kernel. Create a default |
121 | configuration and then run "make menuconfig" to modify it. The end |
122 | result is a .config file that tells the busybox build process what features |
123 | to include. So instead of "./configure; make; make install" the equivalent |
124 | busybox build would be "make defconfig; make; make install". |
125 | </p> |
126 | |
127 | <p> Busybox configured with all features enabled is a little under a megabyte |
128 | dynamically linked on x86. To create a smaller busybox, configure it with |
129 | fewer features. Individual busybox applets cost anywhere from a few |
130 | hundred bytes to tens of kilobytes. Disable unneeded applets to save, |
131 | space, using menuconfig. |
132 | </p> |
133 | |
134 | <p>The most important busybox configurators are:</p> |
135 | |
136 | <ul> |
137 | <li><p>make <b>defconfig</b> - Create the maximum "sane" configuration. This |
138 | enables almost all features, minus things like debugging options and features |
139 | that require changes to the rest of the system to work (such as selinux or |
140 | devfs device names). Use this if you want to start from a full-featured |
141 | busybox and remove features until it's small enough.</p></li> |
142 | <li><p>make <b>allnoconfig</b> - Disable everything. This creates a tiny version |
143 | of busybox that doesn't do anything. Start here if you know exactly what |
144 | you want and would like to select only those features.</p></li> |
145 | <li><p>make <b>menuconfig</b> - Interactively modify a .config file through a |
146 | multi-level menu interface. Use this after one of the previous two.</p></li> |
147 | </ul> |
148 | |
149 | <p>Some other configuration options are:</p> |
150 | <ul> |
151 | <li><p>make <b>oldconfig</b> - Update an old .config file for a newer version |
152 | of busybox.</p></li> |
153 | <li><p>make <b>allyesconfig</b> - Select absolutely everything. This creates |
154 | a statically linked version of busybox full of debug code, with dependencies on |
155 | selinux, using devfs names... This makes sure everything compiles. Whether |
156 | or not the result would do anything useful is an open question.</p></li> |
157 | <li><p>make <b>allbareconfig</b> - Select all applets but disable all sub-features |
158 | within each applet. More build coverage testing.</p></li> |
159 | <li><p>make <b>randconfig</b> - Create a random configuration for test purposes.</p></li> |
160 | </ul> |
161 | |
162 | <p> Menuconfig modifies your .config file through an interactive menu where you can enable or disable |
163 | busybox features, and get help about each feature. |
164 | |
165 | <p> |
166 | To build a smaller busybox binary, run "make menuconfig" and disable the |
167 | features you don't need. (Or run "make allnoconfig" and then use |
168 | menuconfig to add just the features you need. Don't forget to recompile |
169 | with "make" once you've finished configuring.) |
170 | </p> |
171 | |
172 | <hr /> |
173 | <h2><a name="build">How do I build BusyBox with a cross-compiler?</a></h2> |
174 | |
175 | <p> |
176 | To build busybox with a cross-compiler, specify CROSS_COMPILE=<prefix>. |
177 | </p> |
178 | <p> |
179 | CROSS_COMPILE specifies the prefix used for all executables used |
180 | during compilation. Only gcc and related binutils executables |
181 | are prefixed with $(CROSS_COMPILE) in the makefiles. |
182 | CROSS_COMPILE can be set on the command line: |
183 | </p> |
184 | <pre> |
185 | make CROSS_COMPILE=arm-linux-uclibcgnueabi- |
186 | </pre> |
187 | <p> |
188 | Alternatively CROSS_COMPILE can be set in the environment. |
189 | Default value for CROSS_COMPILE is not to prefix executables. |
190 | </p> |
191 | <p> |
192 | To store the cross-compiler in your .config, set the variable |
193 | CONFIG_CROSS_COMPILER_PREFIX accordingly in menuconfig or by |
194 | editing the .config file. |
195 | </p> |
196 | |
197 | <hr /> |
198 | <h2><a name="build_system">How do I build a BusyBox-based system?</a></h2> |
199 | |
200 | <p> |
201 | BusyBox is a package that replaces a dozen standard packages, but it is |
202 | not by itself a complete bootable system. Building an entire Linux |
203 | distribution from source is a bit beyond the scope of this FAQ, but it |
204 | understandably keeps cropping up on the mailing list, so here are some |
205 | pointers. |
206 | </p> |
207 | <p> |
208 | Start by learning how to strip a working system down to the bare essentials |
209 | needed to run one or two commands, so you know what it is you actually |
210 | need. An excellent practical place to do |
211 | this is the <a href="http://www.tldp.org/HOWTO/Bootdisk-HOWTO/">Linux |
212 | BootDisk Howto</a>, or for a more theoretical approach try |
213 | <a href="http://www.tldp.org/HOWTO/From-PowerUp-To-Bash-Prompt-HOWTO.html">From |
214 | PowerUp to Bash Prompt</a>. |
215 | </p> |
216 | <p> |
217 | To learn how to build a working Linux system entirely from source code, |
218 | the place to go is the <a href="http://www.linuxfromscratch.org/">Linux |
219 | From Scratch</a> project. They have an entire book of step-by-step |
220 | instructions you can |
221 | <a href="http://www.linuxfromscratch.org/lfs/view/stable/">read online</a> |
222 | or |
223 | <a href="http://www.linuxfromscratch.org/lfs/downloads/stable/">download</a>. |
224 | Be sure to check out the other sections of their main page, including |
225 | Beyond Linux From Scratch, Hardened Linux From Scratch, their Hints |
226 | directory, and their LiveCD project. (They also have mailing lists which |
227 | are better sources of answers to Linux-system building questions than |
228 | the busybox list.) |
229 | </p> |
230 | <p> |
231 | If you want an automated yet customizable system builder which produces |
232 | a BusyBox and uClibc based system, try |
233 | <a href="http://buildroot.uclibc.org/">buildroot</a>, which is |
234 | another project by the maintainer of the uClibc (Erik Andersen). |
235 | Download the tarball, extract it, unset CC, make. |
236 | For more instructions, see the website. |
237 | </p> |
238 | |
239 | <hr /> |
240 | <h2><a name="kernel">Which Linux kernel versions are supported?</a></h2> |
241 | |
242 | <p> |
243 | Full functionality requires Linux 2.4.x or better. (Earlier versions may |
244 | still work, but are no longer regularly tested.) A large fraction of the |
245 | code should run on just about anything. While the current code is fairly |
246 | Linux specific, it should be fairly easy to port the majority of the code |
247 | to support, say, FreeBSD or Solaris, or Mac OS X, or even Windows (if you |
248 | are into that sort of thing). |
249 | </p> |
250 | |
251 | <hr /> |
252 | <h2><a name="arch">Which architectures does BusyBox run on?</a></h2> |
253 | |
254 | <p> |
255 | BusyBox in general will build on any architecture supported by gcc. |
256 | Kernel module loading for 2.4 Linux kernels is currently |
257 | limited to ARM, CRIS, H8/300, x86, ia64, x86_64, m68k, MIPS, PowerPC, |
258 | S390, SH3/4/5, Sparc, v850e, and x86_64 for 2.4.x kernels. |
259 | </p> |
260 | <p> |
261 | With 2.6.x kernels, module loading support should work on all architectures. |
262 | </p> |
263 | |
264 | <hr /> |
265 | <h2><a name="libc">Which C libraries are supported?</a></h2> |
266 | |
267 | <p> |
268 | On Linux, BusyBox releases are tested against uClibc (0.9.27 or later) and |
269 | glibc (2.2 or later). Both should provide full functionality with busybox, |
270 | and if you find a bug we want to hear about it. |
271 | </p> |
272 | <p> |
273 | Linux-libc5 is no longer maintained (and has no known advantages over |
274 | uClibc), dietlibc is known to have numerous unfixed bugs, and klibc is |
275 | missing too many features to build BusyBox. If you require a small C |
276 | library for Linux, the busybox developers recommend uClibc. |
277 | </p> |
278 | <p> |
279 | Some BusyBox applets have been built and run under a combination |
280 | of newlib and libgloss (see |
281 | <a href="http://www.busybox.net/lists/busybox/2005-March/013759.html">this thread</a>). |
282 | This is still experimental, but may be supported in a future release. |
283 | </p> |
284 | |
285 | <hr /> |
286 | <h2><a name="commercial">Can I include BusyBox as part of the software on my device?</a></h2> |
287 | |
288 | <p> |
289 | Yes. As long as you <a href="http://busybox.net/license.html">fully comply |
290 | with the generous terms of the GPL BusyBox license</a> you can ship BusyBox |
291 | as part of the software on your device. |
292 | </p> |
293 | |
294 | <hr /> |
295 | <h2><a name="external">Where can I find other small utilities since busybox |
296 | does not include the features i want?</a></h2> |
297 | |
298 | <p> |
299 | we maintain such a <a href="tinyutils.html">list</a> on this site! |
300 | </p> |
301 | |
302 | <hr /> |
303 | <h2><a name="demanding">I demand that you to add <favorite feature> right now! How come you don't answer all my questions on the mailing list instantly? I demand that you help me with all of my problems <em>Right Now</em>!</a></h2> |
304 | |
305 | <p> |
306 | You have not paid us a single cent and yet you still have the product of |
307 | many years of our work. We are not your slaves! We work on BusyBox |
308 | because we find it useful and interesting. If you go off flaming us, we |
309 | will ignore you. |
310 | |
311 | <hr /> |
312 | <h2><a name="helpme">I need help with BusyBox! What should I do?</a></h2> |
313 | |
314 | <p> |
315 | If you find that you need help with BusyBox, you can ask for help on the |
316 | BusyBox mailing list at busybox@busybox.net.</p> |
317 | |
318 | <p> In addition to the mailing list, Erik Andersen (andersee), Manuel Nova |
319 | (mjn3), Rob Landley (landley), Mike Frysinger (SpanKY), |
320 | Bernhard Reutner-Fischer (blindvt), and other long-time BusyBox developers |
321 | are known to hang out on the uClibc IRC channel: #uclibc on |
322 | irc.freenode.net. There is a |
323 | <a href="http://ibot.Rikers.org/%23uclibc/">web archive of |
324 | daily logs of the #uclibc IRC channel</a> going back to 2002. |
325 | </p> |
326 | |
327 | <p> |
328 | <b>Please do not send private email to Rob, Erik, Manuel, or the other |
329 | BusyBox contributors asking for private help unless you are planning on |
330 | paying for consulting services.</b> |
331 | </p> |
332 | |
333 | <p> |
334 | When we answer questions on the BusyBox mailing list, it helps everyone |
335 | since people with similar problems in the future will be able to get help |
336 | by searching the mailing list archives. Private help is reserved as a paid |
337 | service. If you need to use private communication, or if you are serious |
338 | about getting timely assistance with BusyBox, you should seriously consider |
339 | paying for consulting services. |
340 | </p> |
341 | |
342 | <hr /> |
343 | <h2><a name="contracts">I need you to add <favorite feature>! Are the BusyBox developers willing to be paid in order to fix bugs or add in <favorite feature>? Are you willing to provide support contracts?</a></h2> |
344 | |
345 | <p> |
346 | Yes we are. The easy way to sponsor a new feature is to post an offer on |
347 | the mailing list to see who's interested. You can also email the project's |
348 | maintainer and ask them to recommend someone. |
349 | </p> |
350 | |
351 | <hr /> |
352 | <h1>Troubleshooting</h1> |
353 | |
354 | <hr /> |
355 | <h2><a name="bugs">I think I found a bug in BusyBox! What should I do?</a></h2> |
356 | |
357 | <p> |
358 | If you simply need help with using or configuring BusyBox, please submit a |
359 | detailed description of your problem to the BusyBox mailing list at <a |
360 | href="mailto:busybox@busybox.net">busybox@busybox.net</a>. |
361 | Please do not send email to individual developers asking |
362 | for private help unless you are planning on paying for consulting services. |
363 | When we answer questions on the BusyBox mailing list, it helps everyone, |
364 | while private answers help only you... |
365 | </p> |
366 | |
367 | <p> |
368 | Bug reports and new feature patches sometimes get lost when posted to the |
369 | mailing list, because the developers of BusyBox are busy people and have |
370 | only so much they can keep in their brains at a time. You can post a |
371 | polite reminder after 2-3 days without offending anybody. If that doesn't |
372 | result in a solution, please use the |
373 | <a href="http://bugs.busybox.net/">BusyBox Bug |
374 | and Patch Tracking System</a> to submit a detailed explanation and we'll |
375 | get to it as soon as we can. |
376 | </p> |
377 | |
378 | <p> |
379 | Note that bugs entered into the bug system without being mentioned on the |
380 | mailing list first may languish there for months before anyone even notices |
381 | them. We generally go through the bug system when preparing for new |
382 | development releases, to see what fell through the cracks while we were |
383 | off writing new features. (It's a fast/unreliable vs slow/reliable thing. |
384 | Saves retransits, but the latency sucks.) |
385 | </p> |
386 | |
387 | <hr /> |
388 | <h2><a name="backporting">I'm using an ancient version from the dawn of time and something's broken. Can you backport fixes for free?</a></h2> |
389 | |
390 | <p>Variants of this one get asked a lot.</p> |
391 | |
392 | <p>The purpose of the BusyBox mailing list is to develop and improve BusyBox, |
393 | and we're happy to respond to our users' needs. But if you're coming to the |
394 | list for free tech support we're going to ask you to upgrade to a current |
395 | version before we try to diagnose your problem.</p> |
396 | |
397 | <p>If you're building BusyBox 0.50 with uClibc 0.9.19 and gcc 1.27 there's a |
398 | fairly large chance that whatever problem you're seeing has already been fixed. |
399 | To get that fix, all you have to do is upgrade to a newer version. If you |
400 | don't at least _try_ that, you're wasting our time.</p> |
401 | |
402 | <p>The volunteers are happy to fix any bugs you point out in the current |
403 | versions because doing so helps everybody and makes the project better. We |
404 | want to make the current version work for you. But diagnosing, debugging, and |
405 | backporting fixes to old versions isn't something we do for free, because it |
406 | doesn't help anybody but you. The cost of volunteer tech support is using a |
407 | reasonably current version of the project.</p> |
408 | |
409 | <p>If you don't want to upgrade, you have the complete source code and thus |
410 | the ability to fix it yourself, or hire a consultant to do it for you. If you |
411 | got your version from a vendor who still supports the older version, they can |
412 | help you. But there are limits as to what the volunteers will feel obliged to |
413 | do for you.</p> |
414 | |
415 | <p>As a rule of thumb, volunteers will generally answer polite questions about |
416 | a given version for about three years after its release before it's so old |
417 | we don't remember the answer off the top of our head. And if you want us to |
418 | put any _effort_ into tracking it down, we want you to put in a little effort |
419 | of your own by confirming it's still a problem with the current version. It's |
420 | also hard for us to fix a problem of yours if we can't reproduce it because |
421 | we don't have any systems running an environment that old.</p> |
422 | |
423 | <p>A consultant will happily set up a special environment just to reproduce |
424 | your problem, and you can always ask on the list if any of the developers |
425 | have consulting rates.</p> |
426 | |
427 | <hr /> |
428 | <h2><a name="init">Busybox init isn't working!</a></h2> |
429 | |
430 | <p> |
431 | Init is the first program that runs, so it might be that no programs are |
432 | working on your new system because of a problem with your cross-compiler, |
433 | kernel, console settings, shared libraries, root filesystem... To rule all |
434 | that out, first build a statically linked version of the following "hello |
435 | world" program with your cross compiler toolchain: |
436 | </p> |
437 | <pre> |
438 | #include <stdio.h> |
439 | |
440 | int main(int argc, char *argv) |
441 | { |
442 | printf("Hello world!\n"); |
443 | sleep(999999999); |
444 | } |
445 | </pre> |
446 | |
447 | <p> |
448 | Now try to boot your device with an "init=" argument pointing to your |
449 | hello world program. Did you see the hello world message? Until you |
450 | do, don't bother messing with busybox init. |
451 | </p> |
452 | |
453 | <p> |
454 | Once you've got it working statically linked, try getting it to work |
455 | dynamically linked. Then read the FAQ entry <a href="#build_system">How |
456 | do I build a BusyBox-based system?</a>, and the |
457 | <a href="/downloads/BusyBox.html#item_init">documentation for BusyBox |
458 | init</a>. |
459 | </p> |
460 | |
461 | <hr /> |
462 | <h2><a name="sed">I can't configure busybox on my system.</a></h2> |
463 | |
464 | <p> |
465 | Configuring Busybox depends on a recent version of sed. Older |
466 | distributions (Red Hat 7.2, Debian 3.0) may not come with a |
467 | usable version. Luckily BusyBox can use its own sed to configure itself, |
468 | although this leads to a bit of a chicken and egg problem. |
469 | You can work around this by hand-configuring busybox to build with just |
470 | sed, then putting that sed in your path to configure the rest of busybox |
471 | with, like so: |
472 | </p> |
473 | |
474 | <pre> |
475 | tar xvjf sources/busybox-x.x.x.tar.bz2 |
476 | cd busybox-x.x.x |
477 | make allnoconfig |
478 | make include/bb_config.h |
479 | echo "CONFIG_SED=y" >> .config |
480 | echo "#undef ENABLE_SED" >> include/bb_config.h |
481 | echo "#define ENABLE_SED 1" >> include/bb_config.h |
482 | make |
483 | mv busybox sed |
484 | export PATH=`pwd`:"$PATH" |
485 | </pre> |
486 | |
487 | <p>Then you can run "make defconfig" or "make menuconfig" normally.</p> |
488 | |
489 | <hr /> |
490 | <h2><a name="job_control">Why do I keep getting "sh: can't access tty; job control turned off" errors? Why doesn't Control-C work within my shell?</a></h2> |
491 | |
492 | <p> |
493 | Job control will be turned off since your shell can not obtain a controlling |
494 | terminal. This typically happens when you run your shell on /dev/console. |
495 | The kernel will not provide a controlling terminal on the /dev/console |
496 | device. Your should run your shell on a normal tty such as tty1 or ttyS0 |
497 | and everything will work perfectly. If you <em>REALLY</em> want your shell |
498 | to run on /dev/console, then you can hack your kernel (if you are into that |
499 | sortof thing) by changing drivers/char/tty_io.c to change the lines where |
500 | it sets "noctty = 1;" to instead set it to "0". I recommend you instead |
501 | run your shell on a real console... |
502 | </p> |
503 | |
504 | <hr /> |
505 | <h1>Misc. questions</h1> |
506 | |
507 | <hr /> |
508 | <h2><a name="tz">How do I change the time zone in busybox?</a></h2> |
509 | |
510 | <p>Busybox has nothing to do with the timezone. Please consult your libc |
511 | documentation. (<a href="http://google.com/search?q=uclibc+glibc+timezone">http://google.com/search?q=uclibc+glibc+timezone</a>).</p> |
512 | |
513 | <hr /> |
514 | <h1>Development</h1> |
515 | |
516 | <hr /> |
517 | <h2><a name="goals">What are the goals of busybox?</a></h2> |
518 | |
519 | <p>Busybox aims to be the smallest and simplest correct implementation of the |
520 | standard Linux command line tools. First and foremost, this means the |
521 | smallest executable size we can manage. We also want to have the simplest |
522 | and cleanest implementation we can manage, be <a href="#standards">standards |
523 | compliant</a>, minimize run-time memory usage (heap and stack), run fast, and |
524 | take over the world.</p> |
525 | |
526 | <hr /> |
527 | <h2><a name="design">What is the design of busybox?</a></h2> |
528 | |
529 | <p>Busybox is like a swiss army knife: one thing with many functions. |
530 | The busybox executable can act like many different programs depending on |
531 | the name used to invoke it. Normal practice is to create a bunch of symlinks |
532 | pointing to the busybox binary, each of which triggers a different busybox |
533 | function. (See <a href="FAQ.html#getting_started">getting started</a> in the |
534 | FAQ for more information on usage, and <a href="BusyBox.html">the |
535 | busybox documentation</a> for a list of symlink names and what they do.) |
536 | |
537 | <p>The "one binary to rule them all" approach is primarily for size reasons: a |
538 | single multi-purpose executable is smaller then many small files could be. |
539 | This way busybox only has one set of ELF headers, it can easily share code |
540 | between different apps even when statically linked, it has better packing |
541 | efficiency by avoding gaps between files or compression dictionary resets, |
542 | and so on.</p> |
543 | |
544 | <p>Work is underway on new options such as "make standalone" to build separate |
545 | binaries for each applet, and a "libbb.so" to make the busybox common code |
546 | available as a shared library. Neither is ready yet at the time of this |
547 | writing.</p> |
548 | |
549 | <a name="source"></a> |
550 | |
551 | <hr /> |
552 | <h2><a name="source_applets">The applet directories</a></h2> |
553 | |
554 | <p>The directory "applets" contains the busybox startup code (applets.c and |
555 | busybox.c), and several subdirectories containing the code for the individual |
556 | applets.</p> |
557 | |
558 | <p>Busybox execution starts with the main() function in applets/busybox.c, |
559 | which sets the global variable applet_name to argv[0] and calls |
560 | run_applet_and_exit() in applets/applets.c. That uses the applets[] array |
561 | (defined in include/busybox.h and filled out in include/applets.h) to |
562 | transfer control to the appropriate APPLET_main() function (such as |
563 | cat_main() or sed_main()). The individual applet takes it from there.</p> |
564 | |
565 | <p>This is why calling busybox under a different name triggers different |
566 | functionality: main() looks up argv[0] in applets[] to get a function pointer |
567 | to APPLET_main().</p> |
568 | |
569 | <p>Busybox applets may also be invoked through the multiplexor applet |
570 | "busybox" (see busybox_main() in libbb/appletlib.c), and through the |
571 | standalone shell (grep for STANDALONE_SHELL in applets/shell/*.c). |
572 | See <a href="FAQ.html#getting_started">getting started</a> in the |
573 | FAQ for more information on these alternate usage mechanisms, which are |
574 | just different ways to reach the relevant APPLET_main() function.</p> |
575 | |
576 | <p>The applet subdirectories (archival, console-tools, coreutils, |
577 | debianutils, e2fsprogs, editors, findutils, init, loginutils, miscutils, |
578 | modutils, networking, procps, shell, sysklogd, and util-linux) correspond |
579 | to the configuration sub-menus in menuconfig. Each subdirectory contains the |
580 | code to implement the applets in that sub-menu, as well as a Config.in |
581 | file defining that configuration sub-menu (with dependencies and help text |
582 | for each applet), and the makefile segment (Makefile.in) for that |
583 | subdirectory.</p> |
584 | |
585 | <p>The run-time --help is stored in usage_messages[], which is initialized at |
586 | the start of applets/applets.c and gets its help text from usage.h. During the |
587 | build this help text is also used to generate the BusyBox documentation (in |
588 | html, txt, and man page formats) in the docs directory. See |
589 | <a href="#adding">adding an applet to busybox</a> for more |
590 | information.</p> |
591 | |
592 | <hr /> |
593 | <h2><a name="source_libbb"><b>libbb</b></a></h2> |
594 | |
595 | <p>Most non-setup code shared between busybox applets lives in the libbb |
596 | directory. It's a mess that evolved over the years without much auditing |
597 | or cleanup. For anybody looking for a great project to break into busybox |
598 | development with, documenting libbb would be both incredibly useful and good |
599 | experience.</p> |
600 | |
601 | <p>Common themes in libbb include allocation functions that test |
602 | for failure and abort the program with an error message so the caller doesn't |
603 | have to test the return value (xmalloc(), xstrdup(), etc), wrapped versions |
604 | of open(), close(), read(), and write() that test for their own failures |
605 | and/or retry automatically, linked list management functions (llist.c), |
606 | command line argument parsing (getopt32.c), and a whole lot more.</p> |
607 | |
608 | <hr /> |
609 | <h2><a name="optimize">I want to make busybox even smaller, how do I go about it?</a></h2> |
610 | |
611 | <p> |
612 | To conserve bytes it's good to know where they're being used, and the |
613 | size of the final executable isn't always a reliable indicator of |
614 | the size of the components (since various structures are rounded up, |
615 | so a small change may not even be visible by itself, but many small |
616 | savings add up). |
617 | </p> |
618 | |
619 | <p> The busybox Makefile builds two versions of busybox, one of which |
620 | (busybox_unstripped) has extra information that various analysis tools |
621 | can use. (This has nothing to do with CONFIG_DEBUG, leave that off |
622 | when trying to optimize for size.) |
623 | </p> |
624 | |
625 | <p> The <b>"make bloatcheck"</b> option uses Matt Mackall's bloat-o-meter |
626 | script to compare two versions of busybox (busybox_unstripped vs |
627 | busybox_old), and report which symbols changed size and by how much. |
628 | To use it, first build a base version with <b>"make baseline"</b>. |
629 | (This creates busybox_old, which should have the original sizes for |
630 | comparison purposes.) Then build the new version with your changes |
631 | and run "make bloatcheck" to see the size differences from the old |
632 | version. |
633 | </p> |
634 | <p> |
635 | The first line of output has totals: how many symbols were added or |
636 | removed, how many symbols grew or shrank, the number of bytes added |
637 | and number of bytes removed by these changes, and finally the total |
638 | number of bytes difference between the two files. The remaining |
639 | lines show each individual symbol, the old and new sizes, and the |
640 | increase or decrease in size (which results are sorted by). |
641 | </p> |
642 | <p> |
643 | The <b>"make sizes"</b> option produces raw symbol size information for |
644 | busybox_unstripped. This is the output from the "nm --size-sort" |
645 | command (see "man nm" for more information), and is the information |
646 | bloat-o-meter parses to produce the comparison report above. For |
647 | defconfig, this is a good way to find the largest symbols in the tree |
648 | (which is a good place to start when trying to shrink the code). To |
649 | take a closer look at individual applets, configure busybox with just |
650 | one applet (run "make allnoconfig" and then switch on a single applet |
651 | with menuconfig), and then use "make sizes" to see the size of that |
652 | applet's components. |
653 | </p> |
654 | <p> |
655 | The "showasm" command (in the scripts directory) produces an assembly |
656 | dump of a function, providing a closer look at what changed. Try |
657 | "scripts/showasm busybox_unstripped" to list available symbols, and |
658 | "scripts/showasm busybox_unstripped symbolname" to see the assembly |
659 | for a sepecific symbol. |
660 | </p> |
661 | |
662 | <hr /> |
663 | <h2><a name="adding">Adding an applet to busybox</a></h2> |
664 | |
665 | <p>To add a new applet to busybox, first pick a name for the applet and |
666 | a corresponding CONFIG_NAME. Then do this:</p> |
667 | |
668 | <ul> |
669 | <li>Figure out where in the busybox source tree your applet best fits, |
670 | and put your source code there. Be sure to use APPLET_main() instead |
671 | of main(), where APPLET is the name of your applet.</li> |
672 | |
673 | <li>Add your applet to the relevant Config.in file (which file you add |
674 | it to determines where it shows up in "make menuconfig"). This uses |
675 | the same general format as the linux kernel's configuration system.</li> |
676 | |
677 | <li>Add your applet to the relevant Makefile.in file (in the same |
678 | directory as the Config.in you chose), using the existing entries as a |
679 | template and the same CONFIG symbol as you used for Config.in. (Don't |
680 | forget "needlibm" or "needcrypt" if your applet needs libm or |
681 | libcrypt.)</li> |
682 | |
683 | <li>Add your applet to "include/applets.h", using one of the existing |
684 | entries as a template. (Note: this is in alphabetical order. Applets |
685 | are found via binary search, and if you add an applet out of order it |
686 | won't work.)</li> |
687 | |
688 | <li>Add your applet's runtime help text to "include/usage.h". You need |
689 | at least appname_trivial_usage (the minimal help text, always included |
690 | in the busybox binary when this applet is enabled) and appname_full_usage |
691 | (extra help text included in the busybox binary with |
692 | CONFIG_FEATURE_VERBOSE_USAGE is enabled), or it won't compile. |
693 | The other two help entry types (appname_example_usage and |
694 | appname_notes_usage) are optional. They don't take up space in the binary, |
695 | but instead show up in the generated documentation (BusyBox.html, |
696 | BusyBox.txt, and the man page BusyBox.1).</li> |
697 | |
698 | <li>Run menuconfig, switch your applet on, compile, test, and fix the |
699 | bugs. Be sure to try both "allyesconfig" and "allnoconfig" (and |
700 | "allbareconfig" if relevant).</li> |
701 | |
702 | </ul> |
703 | |
704 | <hr /> |
705 | <h2><a name="standards">What standards does busybox adhere to?</a></h2> |
706 | |
707 | <p>The standard we're paying attention to is the "Shell and Utilities" |
708 | portion of the <a href="http://www.opengroup.org/onlinepubs/009695399/">Open |
709 | Group Base Standards</a> (also known as the Single Unix Specification version |
710 | 3 or SUSv3). Note that paying attention isn't necessarily the same thing as |
711 | following it.</p> |
712 | |
713 | <p>SUSv3 doesn't even mention things like init, mount, tar, or losetup, nor |
714 | commonly used options like echo's '-e' and '-n', or sed's '-i'. Busybox is |
715 | driven by what real users actually need, not the fact the standard believes |
716 | we should implement ed or sccs. For size reasons, we're unlikely to include |
717 | much internationalization support beyond UTF-8, and on top of all that, our |
718 | configuration menu lets developers chop out features to produce smaller but |
719 | very non-standard utilities.</p> |
720 | |
721 | <p>Also, Busybox is aimed primarily at Linux. Unix standards are interesting |
722 | because Linux tries to adhere to them, but portability to dozens of platforms |
723 | is only interesting in terms of offering a restricted feature set that works |
724 | everywhere, not growing dozens of platform-specific extensions. Busybox |
725 | should be portable to all hardware platforms Linux supports, and any other |
726 | similar operating systems that are easy to do and won't require much |
727 | maintenance.</p> |
728 | |
729 | <p>In practice, standards compliance tends to be a clean-up step once an |
730 | applet is otherwise finished. When polishing and testing a busybox applet, |
731 | we ensure we have at least the option of full standards compliance, or else |
732 | document where we (intentionally) fall short.</p> |
733 | |
734 | <hr /> |
735 | <h2><a name="portability">Portability.</a></h2> |
736 | |
737 | <p>Busybox is a Linux project, but that doesn't mean we don't have to worry |
738 | about portability. First of all, there are different hardware platforms, |
739 | different C library implementations, different versions of the kernel and |
740 | build toolchain... The file "include/platform.h" exists to centralize and |
741 | encapsulate various platform-specific things in one place, so most busybox |
742 | code doesn't have to care where it's running.</p> |
743 | |
744 | <p>To start with, Linux runs on dozens of hardware platforms. We try to test |
745 | each release on x86, x86-64, arm, power pc, and mips. (Since qemu can handle |
746 | all of these, this isn't that hard.) This means we have to care about a number |
747 | of portability issues like endianness, word size, and alignment, all of which |
748 | belong in platform.h. That header handles conditional #includes and gives |
749 | us macros we can use in the rest of our code. At some point in the future |
750 | we might grow a platform.c, possibly even a platform subdirectory. As long |
751 | as the applets themselves don't have to care.</p> |
752 | |
753 | <p>On a related note, we made the "default signedness of char varies" problem |
754 | go away by feeding the compiler -funsigned-char. This gives us consistent |
755 | behavior on all platforms, and defaults to 8-bit clean text processing (which |
756 | gets us halfway to UTF-8 support). NOMMU support is less easily separated |
757 | (see the tips section later in this document), but we're working on it.</p> |
758 | |
759 | <p>Another type of portability is build environments: we unapologetically use |
760 | a number of gcc and glibc extensions (as does the Linux kernel), but these have |
761 | been picked up by packages like uClibc, TCC, and Intel's C Compiler. As for |
762 | gcc, we take advantage of newer compiler optimizations to get the smallest |
763 | possible size, but we also regression test against an older build environment |
764 | using the Red Hat 9 image at "http://busybox.net/downloads/qemu". This has a |
765 | 2.4 kernel, gcc 3.2, make 3.79.1, and glibc 2.3, and is the oldest |
766 | build/deployment environment we still put any effort into maintaining. (If |
767 | anyone takes an interest in older kernels you're welcome to submit patches, |
768 | but the effort would probably be better spent |
769 | <a href="http://www.selenic.com/linux-tiny/">trimming |
770 | down the 2.6 kernel</a>.) Older gcc versions than that are uninteresting since |
771 | we now use c99 features, although |
772 | <a href="http://fabrice.bellard.free.fr/tcc/">tcc</a> might be worth a |
773 | look.</p> |
774 | |
775 | <p>We also test busybox against the current release of uClibc. Older versions |
776 | of uClibc aren't very interesting (they were buggy, and uClibc wasn't really |
777 | usable as a general-purpose C library before version 0.9.26 anyway).</p> |
778 | |
779 | <p>Other unix implementations are mostly uninteresting, since Linux binaries |
780 | have become the new standard for portable Unix programs. Specifically, |
781 | the ubiquity of Linux was cited as the main reason the Intel Binary |
782 | Compatability Standard 2 died, by the standards group organized to name a |
783 | successor to ibcs2: <a href="http://www.telly.org/86open/">the 86open |
784 | project</a>. That project disbanded in 1999 with the endorsement of an |
785 | existing standard: Linux ELF binaries. Since then, the major players at the |
786 | time (such as <a |
787 | href="http://www-03.ibm.com/servers/aix/products/aixos/linux/index.html">AIX</a>, <a |
788 | href="http://www.sun.com/software/solaris/ds/linux_interop.jsp#3">Solaris</a>, and |
789 | <a href="http://www.onlamp.com/pub/a/bsd/2000/03/17/linuxapps.html">FreeBSD</a>) |
790 | have all either grown Linux support or folded.</p> |
791 | |
792 | <p>The major exceptions are newcomer MacOS X, some embedded environments |
793 | (such as newlib+libgloss) which provide a posix environment but not a full |
794 | Linux environment, and environments like Cygwin that provide only partial Linux |
795 | emulation. Also, some embedded Linux systems run a Linux kernel but amputate |
796 | things like the /proc directory to save space.</p> |
797 | |
798 | <p>Supporting these systems is largely a question of providing a clean subset |
799 | of BusyBox's functionality -- whichever applets can easily be made to |
800 | work in that environment. Annotating the configuration system to |
801 | indicate which applets require which prerequisites (such as procfs) is |
802 | also welcome. Other efforts to support these systems (swapping #include |
803 | files to build in different environments, adding adapter code to platform.h, |
804 | adding more extensive special-case supporting infrastructure such as mount's |
805 | legacy mtab support) are handled on a case-by-case basis. Support that can be |
806 | cleanly hidden in platform.h is reasonably attractive, and failing that |
807 | support that can be cleanly separated into a separate conditionally compiled |
808 | file is at least worth a look. Special-case code in the body of an applet is |
809 | something we're trying to avoid.</p> |
810 | |
811 | <hr /> |
812 | <h2><a name="tips">Programming tips and tricks.</a></h2> |
813 | |
814 | <p>Various things busybox uses that aren't particularly well documented |
815 | elsewhere.</p> |
816 | |
817 | <hr /> |
818 | <h2><a name="tips_encrypted_passwords">Encrypted Passwords</a></h2> |
819 | |
820 | <p>Password fields in /etc/passwd and /etc/shadow are in a special format. |
821 | If the first character isn't '$', then it's an old DES style password. If |
822 | the first character is '$' then the password is actually three fields |
823 | separated by '$' characters:</p> |
824 | <pre> |
825 | <b>$type$salt$encrypted_password</b> |
826 | </pre> |
827 | |
828 | <p>The "type" indicates which encryption algorithm to use: 1 for MD5 and 2 for SHA1.</p> |
829 | |
830 | <p>The "salt" is a bunch of ramdom characters (generally 8) the encryption |
831 | algorithm uses to perturb the password in a known and reproducible way (such |
832 | as by appending the random data to the unencrypted password, or combining |
833 | them with exclusive or). Salt is randomly generated when setting a password, |
834 | and then the same salt value is re-used when checking the password. (Salt is |
835 | thus stored unencrypted.)</p> |
836 | |
837 | <p>The advantage of using salt is that the same cleartext password encrypted |
838 | with a different salt value produces a different encrypted value. |
839 | If each encrypted password uses a different salt value, an attacker is forced |
840 | to do the cryptographic math all over again for each password they want to |
841 | check. Without salt, they could simply produce a big dictionary of commonly |
842 | used passwords ahead of time, and look up each password in a stolen password |
843 | file to see if it's a known value. (Even if there are billions of possible |
844 | passwords in the dictionary, checking each one is just a binary search against |
845 | a file only a few gigabytes long.) With salt they can't even tell if two |
846 | different users share the same password without guessing what that password |
847 | is and decrypting it. They also can't precompute the attack dictionary for |
848 | a specific password until they know what the salt value is.</p> |
849 | |
850 | <p>The third field is the encrypted password (plus the salt). For md5 this |
851 | is 22 bytes.</p> |
852 | |
853 | <p>The busybox function to handle all this is pw_encrypt(clear, salt) in |
854 | "libbb/pw_encrypt.c". The first argument is the clear text password to be |
855 | encrypted, and the second is a string in "$type$salt$password" format, from |
856 | which the "type" and "salt" fields will be extracted to produce an encrypted |
857 | value. (Only the first two fields are needed, the third $ is equivalent to |
858 | the end of the string.) The return value is an encrypted password in |
859 | /etc/passwd format, with all three $ separated fields. It's stored in |
860 | a static buffer, 128 bytes long.</p> |
861 | |
862 | <p>So when checking an existing password, if pw_encrypt(text, |
863 | old_encrypted_password) returns a string that compares identical to |
864 | old_encrypted_password, you've got the right password. When setting a new |
865 | password, generate a random 8 character salt string, put it in the right |
866 | format with sprintf(buffer, "$%c$%s", type, salt), and feed buffer as the |
867 | second argument to pw_encrypt(text,buffer).</p> |
868 | |
869 | <hr /> |
870 | <h2><a name="tips_vfork">Fork and vfork</a></h2> |
871 | |
872 | <p>On systems that haven't got a Memory Management Unit, fork() is unreasonably |
873 | expensive to implement (and sometimes even impossible), so a less capable |
874 | function called vfork() is used instead. (Using vfork() on a system with an |
875 | MMU is like pounding a nail with a wrench. Not the best tool for the job, but |
876 | it works.)</p> |
877 | |
878 | <p>Busybox hides the difference between fork() and vfork() in |
879 | libbb/bb_fork_exec.c. If you ever want to fork and exec, use bb_fork_exec() |
880 | (which returns a pid and takes the same arguments as execve(), although in |
881 | this case envp can be NULL) and don't worry about it. This description is |
882 | here in case you want to know why that does what it does.</p> |
883 | |
884 | <p>Implementing fork() depends on having a Memory Management Unit. With an |
885 | MMU then you can simply set up a second set of page tables and share the |
886 | physical memory via copy-on-write. So a fork() followed quickly by exec() |
887 | only copies a few pages of the parent's memory, just the ones it changes |
888 | before freeing them.</p> |
889 | |
890 | <p>With a very primitive MMU (using a base pointer plus length instead of page |
891 | tables, which can provide virtual addresses and protect processes from each |
892 | other, but no copy on write) you can still implement fork. But it's |
893 | unreasonably expensive, because you have to copy all the parent process' |
894 | memory into the new process (which could easily be several megabytes per fork). |
895 | And you have to do this even though that memory gets freed again as soon as the |
896 | exec happens. (This is not just slow and a waste of space but causes memory |
897 | usage spikes that can easily cause the system to run out of memory.)</p> |
898 | |
899 | <p>Without even a primitive MMU, you have no virtual addresses. Every process |
900 | can reach out and touch any other process' memory, because all pointers are to |
901 | physical addresses with no protection. Even if you copy a process' memory to |
902 | new physical addresses, all of its pointers point to the old objects in the |
903 | old process. (Searching through the new copy's memory for pointers and |
904 | redirect them to the new locations is not an easy problem.)</p> |
905 | |
906 | <p>So with a primitive or missing MMU, fork() is just not a good idea.</p> |
907 | |
908 | <p>In theory, vfork() is just a fork() that writeably shares the heap and stack |
909 | rather than copying it (so what one process writes the other one sees). In |
910 | practice, vfork() has to suspend the parent process until the child does exec, |
911 | at which point the parent wakes up and resumes by returning from the call to |
912 | vfork(). All modern kernel/libc combinations implement vfork() to put the |
913 | parent to sleep until the child does its exec. There's just no other way to |
914 | make it work: the parent has to know the child has done its exec() or exit() |
915 | before it's safe to return from the function it's in, so it has to block |
916 | until that happens. In fact without suspending the parent there's no way to |
917 | even store separate copies of the return value (the pid) from the vfork() call |
918 | itself: both assignments write into the same memory location.</p> |
919 | |
920 | <p>One way to understand (and in fact implement) vfork() is this: imagine |
921 | the parent does a setjmp and then continues on (pretending to be the child) |
922 | until the exec() comes around, then the _exec_ does the actual fork, and the |
923 | parent does a longjmp back to the original vfork call and continues on from |
924 | there. (It thus becomes obvious why the child can't return, or modify |
925 | local variables it doesn't want the parent to see changed when it resumes.) |
926 | |
927 | <p>Note a common mistake: the need for vfork doesn't mean you can't have two |
928 | processes running at the same time. It means you can't have two processes |
929 | sharing the same memory without stomping all over each other. As soon as |
930 | the child calls exec(), the parent resumes.</p> |
931 | |
932 | <p>If the child's attempt to call exec() fails, the child should call _exit() |
933 | rather than a normal exit(). This avoids any atexit() code that might confuse |
934 | the parent. (The parent should never call _exit(), only a vforked child that |
935 | failed to exec.)</p> |
936 | |
937 | <p>(Now in theory, a nommu system could just copy the _stack_ when it forks |
938 | (which presumably is much shorter than the heap), and leave the heap shared. |
939 | Even with no MMU at all |
940 | In practice, you've just wound up in a multi-threaded situation and you can't |
941 | do a malloc() or free() on your heap without freeing the other process' memory |
942 | (and if you don't have the proper locking for being threaded, corrupting the |
943 | heap if both of you try to do it at the same time and wind up stomping on |
944 | each other while traversing the free memory lists). The thing about vfork is |
945 | that it's a big red flag warning "there be dragons here" rather than |
946 | something subtle and thus even more dangerous.)</p> |
947 | |
948 | <hr /> |
949 | <h2><a name="tips_sort_read">Short reads and writes</a></h2> |
950 | |
951 | <p>Busybox has special functions, bb_full_read() and bb_full_write(), to |
952 | check that all the data we asked for got read or written. Is this a real |
953 | world consideration? Try the following:</p> |
954 | |
955 | <pre>while true; do echo hello; sleep 1; done | tee out.txt</pre> |
956 | |
957 | <p>If tee is implemented with bb_full_read(), tee doesn't display output |
958 | in real time but blocks until its entire input buffer (generally a couple |
959 | kilobytes) is read, then displays it all at once. In that case, we _want_ |
960 | the short read, for user interface reasons. (Note that read() should never |
961 | return 0 unless it has hit the end of input, and an attempt to write 0 |
962 | bytes should be ignored by the OS.)</p> |
963 | |
964 | <p>As for short writes, play around with two processes piping data to each |
965 | other on the command line (cat bigfile | gzip > out.gz) and suspend and |
966 | resume a few times (ctrl-z to suspend, "fg" to resume). The writer can |
967 | experience short writes, which are especially dangerous because if you don't |
968 | notice them you'll discard data. They can also happen when a system is under |
969 | load and a fast process is piping to a slower one. (Such as an xterm waiting |
970 | on x11 when the scheduler decides X is being a CPU hog with all that |
971 | text console scrolling...)</p> |
972 | |
973 | <p>So will data always be read from the far end of a pipe at the |
974 | same chunk sizes it was written in? Nope. Don't rely on that. For one |
975 | counterexample, see <a href="http://www.faqs.org/rfcs/rfc896.html">rfc 896 |
976 | for Nagle's algorithm</a>, which waits a fraction of a second or so before |
977 | sending out small amounts of data through a TCP/IP connection in case more |
978 | data comes in that can be merged into the same packet. (In case you were |
979 | wondering why action games that use TCP/IP set TCP_NODELAY to lower the latency |
980 | on their their sockets, now you know.)</p> |
981 | |
982 | <hr /> |
983 | <h2><a name="tips_memory">Memory used by relocatable code, PIC, and static linking.</a></h2> |
984 | |
985 | <p>The downside of standard dynamic linking is that it results in self-modifying |
986 | code. Although each executable's pages are mmaped() into a process' address |
987 | space from the executable file and are thus naturally shared between processes |
988 | out of the page cache, the library loader (ld-linux.so.2 or ld-uClibc.so.0) |
989 | writes to these pages to supply addresses for relocatable symbols. This |
990 | dirties the pages, triggering copy-on-write allocation of new memory for each |
991 | processes' dirtied pages.</p> |
992 | |
993 | <p>One solution to this is Position Independent Code (PIC), a way of linking |
994 | a file so all the relocations are grouped together. This dirties fewer |
995 | pages (often just a single page) for each process' relocations. The down |
996 | side is this results in larger executables, which take up more space on disk |
997 | (and a correspondingly larger space in memory). But when many copies of the |
998 | same program are running, PIC dynamic linking trades a larger disk footprint |
999 | for a smaller memory footprint, by sharing more pages.</p> |
1000 | |
1001 | <p>A third solution is static linking. A statically linked program has no |
1002 | relocations, and thus the entire executable is shared between all running |
1003 | instances. This tends to have a significantly larger disk footprint, but |
1004 | on a system with only one or two executables, shared libraries aren't much |
1005 | of a win anyway.</p> |
1006 | |
1007 | <p>You can tell the glibc linker to display debugging information about its |
1008 | relocations with the environment variable "LD_DEBUG". Try |
1009 | "LD_DEBUG=help /bin/true" for a list of commands. Learning to interpret |
1010 | "LD_DEBUG=statistics cat /proc/self/statm" could be interesting.</p> |
1011 | |
1012 | <p>For more on this topic, here's Rich Felker:</p> |
1013 | <blockquote> |
1014 | <p>Dynamic linking (without fixed load addresses) fundamentally requires |
1015 | at least one dirty page per dso that uses symbols. Making calls (but |
1016 | never taking the address explicitly) to functions within the same dso |
1017 | does not require a dirty page by itself, but will with ELF unless you |
1018 | use -Bsymbolic or hidden symbols when linking.</p> |
1019 | |
1020 | <p>ELF uses significant additional stack space for the kernel to pass all |
1021 | the ELF data structures to the newly created process image. These are |
1022 | located above the argument list and environment. This normally adds 1 |
1023 | dirty page to the process size.</p> |
1024 | |
1025 | <p>The ELF dynamic linker has its own data segment, adding one or more |
1026 | dirty pages. I believe it also performs relocations on itself.</p> |
1027 | |
1028 | <p>The ELF dynamic linker makes significant dynamic allocations to manage |
1029 | the global symbol table and the loaded dso's. This data is never |
1030 | freed. It will be needed again if libdl is used, so unconditionally |
1031 | freeing it is not possible, but normal programs do not use libdl. Of |
1032 | course with glibc all programs use libdl (due to nsswitch) so the |
1033 | issue was never addressed.</p> |
1034 | |
1035 | <p>ELF also has the issue that segments are not page-aligned on disk. |
1036 | This saves up to 4k on disk, but at the expense of using an additional |
1037 | dirty page in most cases, due to a large portion of the first data |
1038 | page being filled with a duplicate copy of the last text page.</p> |
1039 | |
1040 | <p>The above is just a partial list of the tiny memory penalties of ELF |
1041 | dynamic linking, which eventually add up to quite a bit. The smallest |
1042 | I've been able to get a process down to is 8 dirty pages, and the |
1043 | above factors seem to mostly account for it (but some were difficult |
1044 | to measure).</p> |
1045 | </blockquote> |
1046 | |
1047 | <hr /> |
1048 | <h2><a name="tips_kernel_headers"></a>Including kernel headers</h2> |
1049 | |
1050 | <p>The "linux" or "asm" directories of /usr/include |
1051 | contain Linux kernel |
1052 | headers, so that the C library can talk directly to the Linux kernel. In |
1053 | a perfect world, applications shouldn't include these headers directly, but |
1054 | we don't live in a perfect world.</p> |
1055 | |
1056 | <p>For example, Busybox's losetup code wants linux/loop.c because nothing else |
1057 | #defines the structures to call the kernel's loopback device setup ioctls. |
1058 | Attempts to cut and paste the information into a local busybox header file |
1059 | proved incredibly painful, because portions of the loop_info structure vary by |
1060 | architecture, namely the type __kernel_dev_t has different sizes on alpha, |
1061 | arm, x86, and so on. Meaning we either #include <linux/posix_types.h> or |
1062 | we hardwire #ifdefs to check what platform we're building on and define this |
1063 | type appropriately for every single hardware architecture supported by |
1064 | Linux, which is simply unworkable.</p> |
1065 | |
1066 | <p>This is aside from the fact that the relevant type defined in |
1067 | posix_types.h was renamed to __kernel_old_dev_t during the 2.5 series, so |
1068 | to cut and paste the structure into our header we have to #include |
1069 | <linux/version.h> to figure out which name to use. (What we actually |
1070 | do is |
1071 | check if we're building on 2.6, and if so just use the new 64 bit structure |
1072 | instead to avoid the rename entirely.) But we still need the version |
1073 | check, since 2.4 didn't have the 64 bit structure.</p> |
1074 | |
1075 | <p>The BusyBox developers spent <u>two years</u> trying to figure |
1076 | out a clean way to do all this. There isn't one. The losetup in the |
1077 | util-linux package from kernel.org isn't doing it cleanly either, they just |
1078 | hide the ugliness by nesting #include files. Their mount/loop.h |
1079 | #includes "my_dev_t.h", which #includes <linux/posix_types.h> |
1080 | and <linux/version.h> just like we do. There simply is no alternative. |
1081 | </p> |
1082 | |
1083 | <p>Just because directly #including kernel headers is sometimes |
1084 | unavoidable doesn't me we should include them when there's a better |
1085 | way to do it. However, block copying information out of the kernel headers |
1086 | is not a better way.</p> |
1087 | |
1088 | <hr /> |
1089 | <h2><a name="who">Who are the BusyBox developers?</a></h2> |
1090 | |
1091 | <p>The following login accounts currently exist on busybox.net. (I.E. these |
1092 | people can commit <a href="http://busybox.net/downloads/patches/">patches</a> |
1093 | into subversion for the BusyBox, uClibc, and buildroot projects.)</p> |
1094 | |
1095 | <pre> |
1096 | aldot :Bernhard Reutner-Fischer |
1097 | andersen :Erik Andersen - uClibc and BuildRoot maintainer. |
1098 | bug1 :Glenn McGrath |
1099 | davidm :David McCullough |
1100 | gkajmowi :Garrett Kajmowicz - uClibc++ maintainer |
1101 | jbglaw :Jan-Benedict Glaw |
1102 | jocke :Joakim Tjernlund |
1103 | landley :Rob Landley |
1104 | lethal :Paul Mundt |
1105 | mjn3 :Manuel Novoa III |
1106 | osuadmin :osuadmin |
1107 | pgf :Paul Fox |
1108 | pkj :Peter Kjellerstedt |
1109 | prpplague :David Anders |
1110 | psm :Peter S. Mazinger |
1111 | russ :Russ Dill |
1112 | sandman :Robert Griebl |
1113 | sjhill :Steven J. Hill |
1114 | solar :Ned Ludd |
1115 | timr :Tim Riker |
1116 | tobiasa :Tobias Anderberg |
1117 | vapier :Mike Frysinger |
1118 | vda :Denys Vlasenko - BusyBox maintainer |
1119 | </pre> |
1120 | |
1121 | <p>The following accounts used to exist on busybox.net, but don't anymore so |
1122 | I can't ask /etc/passwd for their names. Rob Wentworth |
1123 | <robwen at gmail.com> asked Google and recovered the names:</p> |
1124 | |
1125 | <pre> |
1126 | aaronl :Aaron Lehmann |
1127 | beppu :John Beppu |
1128 | dwhedon :David Whedon |
1129 | erik :Erik Andersen |
1130 | gfeldman :Gennady Feldman |
1131 | jimg :Jim Gleason |
1132 | kraai :Matt Kraai |
1133 | markw :Mark Whitley |
1134 | miles :Miles Bader |
1135 | proski :Pavel Roskin |
1136 | rjune :Richard June |
1137 | tausq :Randolph Chung |
1138 | vodz :Vladimir N. Oleynik |
1139 | </pre> |
1140 | |
1141 | |
1142 | <br> |
1143 | <br> |
1144 | <br> |
1145 | |
1146 | <!--#include file="footer.html" --> |