Contents of /tags/mkinitrd-6_2_1/busybox/TODO
Parent Directory | Revision Log
Revision 999 -
(show annotations)
(download)
Sun May 30 12:16:23 2010 UTC (14 years, 4 months ago) by niro
File size: 15233 byte(s)
Sun May 30 12:16:23 2010 UTC (14 years, 4 months ago) by niro
File size: 15233 byte(s)
tagged 'mkinitrd-6_2_1'
1 | Busybox TODO |
2 | |
3 | Stuff that needs to be done. This is organized by who plans to get around to |
4 | doing it eventually, but that doesn't mean they "own" the item. If you want to |
5 | do one of these bounce an email off the person it's listed under to see if they |
6 | have any suggestions how they plan to go about it, and to minimize conflicts |
7 | between your work and theirs. But otherwise, all of these are fair game. |
8 | |
9 | Rob Landley suggested this: |
10 | Implement bb_realpath() that can handle NULL on non-glibc. |
11 | |
12 | Remove obsolete _() wrapper crud for internationalization we don't do. |
13 | Figure out where we need utf8 support, and add it. |
14 | |
15 | sh |
16 | The command shell situation is a mess. We have two different |
17 | shells that don't really share any code, and the "standalone shell" doesn't |
18 | work all that well (especially not in a chroot environment), due to apps not |
19 | being reentrant. |
20 | |
21 | Do a SUSv3 audit |
22 | Look at the full Single Unix Specification version 3 (available online at |
23 | "http://www.opengroup.org/onlinepubs/009695399/nfindex.html") and |
24 | figure out which of our apps are compliant, and what we're missing that |
25 | we might actually care about. |
26 | |
27 | Even better would be some kind of automated compliance test harness that |
28 | exercises each command line option and the various corner cases. |
29 | |
30 | Internationalization |
31 | How much internationalization should we do? |
32 | |
33 | The low hanging fruit is UTF-8 character set support. We should do this. |
34 | (Vodz pointed out the shell's cmdedit as needing work here. What else?) |
35 | |
36 | We also have lots of hardwired english text messages. Consolidating this |
37 | into some kind of message table not only makes translation easier, but |
38 | also allows us to consolidate redundant (or close) strings. |
39 | |
40 | We probably don't want to be bloated with locale support. (Not unless we |
41 | can cleanly export it from our underlying C library without having to |
42 | concern ourselves with it directly. Perhaps a few specific things like a |
43 | config option for "date" are low hanging fruit here?) |
44 | |
45 | What level should things happen at? How much do we care about |
46 | internationalizing the text console when X11 and xterms are so much better |
47 | at it? (There's some infrastructure here we don't implement: The |
48 | "unicode_start" and "unicode_stop" shell scripts need "vt-is-UTF8" and a |
49 | --unicode option to loadkeys. That implies a real loadkeys/dumpkeys |
50 | implementation to replace loadkmap/dumpkmap. Plus messing with console font |
51 | loading. Is it worth it, or do we just say "use X"?) |
52 | |
53 | Individual compilation of applets. |
54 | It would be nice if busybox had the option to compile to individual applets, |
55 | for people who want an alternate implementation less bloated than the gnu |
56 | utils (or simply with less political baggage), but without it being one big |
57 | executable. |
58 | |
59 | Turning libbb into a real dll is another possibility, especially if libbb |
60 | could export some of the other library interfaces we've already more or less |
61 | got the code for (like zlib). |
62 | buildroot - Make a "dogfood" option |
63 | Busybox 1.1 will be capable of replacing most gnu packages for real world |
64 | use, such as developing software or in a live CD. It needs wider testing. |
65 | |
66 | Busybox should now be able to replace bzip2, coreutils, e2fsprogs, file, |
67 | findutils, gawk, grep, inetutils, less, modutils, net-tools, patch, procps, |
68 | sed, shadow, sysklogd, sysvinit, tar, util-linux, and vim. The resulting |
69 | system should be self-hosting (I.E. able to rebuild itself from source |
70 | code). This means it would need (at least) binutils, gcc, and make, or |
71 | equivalents. |
72 | |
73 | It would be a good "eating our own dogfood" test if buildroot had the option |
74 | of using a "make allyesconfig" busybox instead of the all of the above |
75 | packages. Anything that's wrong with the resulting system, we can fix. (It |
76 | would be nice to be able to upgrade busybox to be able to replace bash and |
77 | diffutils as well, but we're not there yet.) |
78 | |
79 | One example of an existing system that does this already is Firmware Linux: |
80 | http://www.landley.net/code/firmware |
81 | initramfs |
82 | Busybox should have a sample initramfs build script. This depends on |
83 | bbsh, mdev, and switch_root. |
84 | mkdep |
85 | Write a mkdep that doesn't segfault if there's a directory it doesn't |
86 | have permission to read, isn't based on manually editing the output of |
87 | lexx and yacc, doesn't make such a mess under include/config, etc. |
88 | Group globals into unions of structures. |
89 | Go through and turn all the global and static variables into structures, |
90 | and have all those structures be in a big union shared between processes, |
91 | so busybox uses less bss. (This is a big win on nommu machines.) See |
92 | sed.c and mdev.c for examples. |
93 | Go through bugs.busybox.net and close out all of that somehow. |
94 | This one's open to everybody, but I'll wind up doing it... |
95 | |
96 | |
97 | Bernhard Reutner-Fischer <busybox@busybox.net> suggests to look at these: |
98 | New debug options: |
99 | -Wlarger-than-127 |
100 | Cleanup any big users |
101 | Collate BUFSIZ IOBUF_SIZE MY_BUF_SIZE PIPE_PROGRESS_SIZE BUFSIZE PIPESIZE |
102 | make bb_common_bufsiz1 configurable, size wise. |
103 | make pipesize configurable, size wise. |
104 | Use bb_common_bufsiz1 throughout applets! |
105 | |
106 | As yet unclaimed: |
107 | |
108 | ---- |
109 | diff |
110 | Make sure we handle empty files properly: |
111 | From the patch man page: |
112 | |
113 | you can remove a file by sending out a context diff that compares |
114 | the file to be deleted with an empty file dated the Epoch. The |
115 | file will be removed unless patch is conforming to POSIX and the |
116 | -E or --remove-empty-files option is not given. |
117 | --- |
118 | patch |
119 | Should have simple fuzz factor support to apply patches at an offset which |
120 | shouldn't take up too much space. |
121 | |
122 | And while we're at it, a new patch filename quoting format is apparently |
123 | coming soon: http://marc.theaimsgroup.com/?l=git&m=112927316408690&w=2 |
124 | --- |
125 | ar |
126 | Write support! |
127 | --- |
128 | stty / catv |
129 | stty's visible() function and catv's guts are identical. Merge them into |
130 | an appropriate libbb function. |
131 | --- |
132 | struct suffix_mult |
133 | Several duplicate users of: grep -r "1024\*1024" * -B2 -A1 |
134 | Merge to a single size_suffixes[] in libbb. |
135 | Users: head tail od_bloaty hexdump and (partially as it wouldn't hurt) svlogd |
136 | --- |
137 | tail |
138 | ./busybox tail -f foo.c~ TODO |
139 | should not print fmt=header_fmt for subsequent date >> TODO; i.e. only |
140 | fmt+ if another (not the current) file did change |
141 | |
142 | Architectural issues: |
143 | |
144 | bb_close() with fsync() |
145 | We should have a bb_close() in place of normal close, with a CONFIG_ option |
146 | to not just check the return value of close() for an error, but fsync(). |
147 | Close can't reliably report anything useful because if write() accepted the |
148 | data then it either went out to the network or it's in cache or a pipe |
149 | buffer. Either way, there's no guarantee it'll make it to its final |
150 | destination before close() gets called, so there's no guarantee that any |
151 | error will be reported. |
152 | |
153 | You need to call fsync() if you care about errors that occur after write(), |
154 | but that can have a big performance impact. So make it a config option. |
155 | --- |
156 | Unify archivers |
157 | Lots of archivers have the same general infrastructure. The directory |
158 | traversal code should be factored out, and the guts of each archiver could |
159 | be some setup code and a series of callbacks for "add this file", |
160 | "add this directory", "add this symlink" and so on. |
161 | |
162 | This could clean up tar and zip, and make it cheaper to add cpio and ar |
163 | write support, and possibly even cheaply add things like mkisofs or |
164 | mksquashfs someday, if they become relevant. |
165 | --- |
166 | Text buffer support. |
167 | Several existing applets (sort, vi, less...) read |
168 | a whole file into memory and act on it. Use open_read_close(). |
169 | --- |
170 | Memory Allocation |
171 | We have a CONFIG_BUFFER mechanism that lets us select whether to do memory |
172 | allocation on the stack or the heap. Unfortunately, we're not using it much. |
173 | We need to audit our memory allocations and turn a lot of malloc/free calls |
174 | into RESERVE_CONFIG_BUFFER/RELEASE_CONFIG_BUFFER. |
175 | For a start, see e.g. make EXTRA_CFLAGS=-Wlarger-than-64 |
176 | |
177 | And while we're at it, many of the CONFIG_FEATURE_CLEAN_UP #ifdefs will be |
178 | optimized out by the compiler in the stack allocation case (since there's no |
179 | free for an alloca()), and this means that various cleanup loops that just |
180 | call free might also be optimized out by the compiler if written right, so |
181 | we can yank those #ifdefs too, and generally clean up the code. |
182 | --- |
183 | Switch CONFIG_SYMBOLS to ENABLE_SYMBOLS |
184 | |
185 | In busybox 1.0 and earlier, configuration was done by CONFIG_SYMBOLS |
186 | that were either defined or undefined to indicate whether the symbol was |
187 | selected in the .config file. They were used with #ifdefs, ala: |
188 | |
189 | #ifdef CONFIG_SYMBOL |
190 | if (other_test) { |
191 | do_code(); |
192 | } |
193 | #endif |
194 | |
195 | In 1.1, we have new ENABLE_SYMBOLS which are always defined (as 0 or 1), |
196 | meaning you can still use them for preprocessor tests by replacing |
197 | "#ifdef CONFIG_SYMBOL" with "#if ENABLE_SYMBOL". But more importantly, we |
198 | can use them as a true or false test in normal C code: |
199 | |
200 | if (ENABLE_SYMBOL && other_test) { |
201 | do_code(); |
202 | } |
203 | |
204 | (Optimizing away if() statements that resolve to a constant value |
205 | is known as "dead code elimination", an optimization so old and simple that |
206 | Turbo Pascal for DOS did it twenty years ago. Even modern mini-compilers |
207 | like the Tiny C Compiler (tcc) and the Small Device C Compiler (SDCC) |
208 | perform dead code elimination.) |
209 | |
210 | Right now, busybox.h is #including both "config.h" (defining the |
211 | CONFIG_SYMBOLS) and "bb_config.h" (defining the ENABLE_SYMBOLS). At some |
212 | point in the future, it would be nice to wean ourselves off of the |
213 | CONFIG versions. (Among other things, some defective build environments |
214 | leak the Linux kernel's CONFIG_SYMBOLS into the system's standard #include |
215 | files. We've experienced collisions before.) |
216 | --- |
217 | FEATURE_CLEAN_UP |
218 | This is more an unresolved issue than a to-do item. More thought is needed. |
219 | |
220 | Normally we rely on exit() to free memory, close files and unmap segments |
221 | for us. This makes most calls to free(), close(), and unmap() optional in |
222 | busybox applets that don't intend to run for very long, and optional stuff |
223 | can be omitted to save size. |
224 | |
225 | The idea was raised that we could simulate fork/exit with setjmp/longjmp |
226 | for _really_ brainless embedded systems, or speed up the standalone shell |
227 | by not forking. Doing so would require a reliable FEATURE_CLEAN_UP. |
228 | Unfortunately, this isn't as easy as it sounds. |
229 | |
230 | The problem is, lots of things exit(), sometimes unexpectedly (xmalloc()) |
231 | and sometimes reliably (bb_perror_msg_and_die() or show_usage()). This |
232 | jumps out of the normal flow control and bypasses any cleanup code we |
233 | put at the end of our applets. |
234 | |
235 | It's possible to add hooks to libbb functions like xmalloc() and xopen() |
236 | to add their entries to a linked list, which could be traversed and |
237 | freed/closed automatically. (This would need to be able to free just the |
238 | entries after a checkpoint to be usable for a forkless standalone shell. |
239 | You don't want to free the shell's own resources.) |
240 | |
241 | Right now, FEATURE_CLEAN_UP is more or less a debugging aid, to make things |
242 | like valgrind happy. It's also documentation of _what_ we're trusting |
243 | exit() to clean up for us. But new infrastructure to auto-free stuff would |
244 | render the existing FEATURE_CLEAN_UP code redundant. |
245 | |
246 | For right now, exit() handles it just fine. |
247 | |
248 | |
249 | Minor stuff: |
250 | watchdog.c could autodetect the timer duration via: |
251 | if(!ioctl (fd, WDIOC_GETTIMEOUT, &tmo)) timer_duration = 1 + (tmo / 2); |
252 | Unfortunately, that needs linux/watchdog.h and that contains unfiltered |
253 | kernel types on some distros, which breaks the build. |
254 | --- |
255 | use bb_error_msg where appropriate: See |
256 | egrep "(printf.*\([[:space:]]*(stderr|2)|[^_]write.*\([[:space:]]*(stderr|2))" |
257 | --- |
258 | use bb_perror_msg where appropriate: See |
259 | egrep "[^_]perror" |
260 | --- |
261 | possible code duplication ingroup() and is_a_group_member() |
262 | --- |
263 | Move __get_hz() to a better place and (re)use it in route.c, ash.c |
264 | --- |
265 | See grep -r strtod |
266 | Alot of duplication that wants cleanup. |
267 | --- |
268 | in_ether duplicated in network/{interface,ifconfig}.c |
269 | --- |
270 | unify progress_meter. wget, flash_eraseall, pipe_progress, fbsplash, setfiles. |
271 | --- |
272 | support start-stop-daemon -d <chdir-path> |
273 | |
274 | Code cleanup: |
275 | |
276 | Replace deprecated functions. |
277 | |
278 | --- |
279 | vdprintf() -> similar sized functionality |
280 | --- |
281 | |
282 | (TODO list after discussion 11.05.2009) |
283 | |
284 | * shrink tc/brctl/ip |
285 | tc/brctl seem like fairly large things to try and tackle in your timeframe, |
286 | and i think people have posted attempts in the past. Adding additional |
287 | options to ip though seems reasonable. |
288 | |
289 | * add tests for some applets |
290 | |
291 | * implement POSIX utilities and audit them for POSIX conformance. then |
292 | audit them for GNU conformance. then document all your findings in a new |
293 | doc/conformance.txt file while perhaps implementing some of the missing |
294 | features. |
295 | you can find the latest POSIX documentation (1003.1-2008) here: |
296 | http://www.opengroup.org/onlinepubs/9699919799/ |
297 | and the complete list of all utilities that POSIX covers: |
298 | http://www.opengroup.org/onlinepubs/9699919799/idx/utilities.html |
299 | The first step would to generate a file/matrix what is already archived |
300 | (also IPV6) |
301 | |
302 | * ntpdate/ntpd (see ntpclient and openntp for examples) |
303 | |
304 | * implement 'at' |
305 | |
306 | * rpcbind (former portmap) or equivalent |
307 | so that we don't have to use -o nolock on nfs mounts |
308 | |
309 | * check IPV6 compliance |
310 | |
311 | * generate a mini example using kernel+busybox only (+libc) for example |
312 | |
313 | * more support for advanced linux 2.6.x features, see: iotop |
314 | most likely there is more |
315 | |
316 | * even more support for statistics: mpstat, iostat, powertop.... |
317 | |
318 | |
319 | Unicode work needed: |
320 | |
321 | Unicode support uses libc multibyte functions if LOCALE_SUPPORT is on |
322 | (in this case, the code will also support many more encodings), |
323 | or uses a limited subset of re-implemented multibyte functions |
324 | which only understand "one byte == one char" and unicode. |
325 | This is useful if you build against uclibc with locale support disabled. |
326 | |
327 | Unicode-dependent applets must call check_unicode_in_env() when they |
328 | begin executing. |
329 | |
330 | Applet code may conditionalize on FEATURE_ASSUME_UNICODE |
331 | in order to use more efficient code if unicode support is not requested. |
332 | |
333 | Available functions (if you need more, implement them in libbb/unicode.c |
334 | so that they work without LOCALE_SUPPORT too): |
335 | |
336 | int bb_mbstrlen(str) - multibyte-aware strlen |
337 | size_t mbstowcs(wdest, src, n) |
338 | size_t wcstombs(dest, wsrc, n) |
339 | size_t wcrtomb(str, wc, wstate) |
340 | int iswspace(wc) |
341 | int iswalnum(wc) |
342 | int iswpunct(wc) |
343 | |
344 | Applets which only need to align columns on screen correctly: |
345 | |
346 | ls - already done, use source as an example |
347 | df |
348 | dumpleases |
349 | lsmod |
350 | |
351 | Applets which need to account for Unicode chars |
352 | while processing the output: |
353 | |
354 | [un]expand |
355 | fold |
356 | man |
357 | watch |
358 | cut (-b and -c are currently the same, needs fixing) |
359 | |
360 | These applets need to ensure that unicode input |
361 | is handled correctly (say, <unicode><backspace> sequence): |
362 | |
363 | getty, login |
364 | rm -i |
365 | unzip (overwrite prompt) |
366 | |
367 | Viewers/editors are more difficult (many cases to get right). |
368 | libbb/lineedit.c is an example how to do it: |
369 | |
370 | less, most, ed, vi |
371 | awk |
372 | [ef]grep |
373 | sed |
374 | |
375 | Probably needs some specialized work: |
376 | |
377 | loadkeys |