Annotation of /tags/mkinitrd-6_5_2/busybox/TODO
Parent Directory | Revision Log
Revision 1522 -
(hide annotations)
(download)
Wed Sep 7 18:40:16 2011 UTC (13 years ago) by niro
File size: 12079 byte(s)
Wed Sep 7 18:40:16 2011 UTC (13 years ago) by niro
File size: 12079 byte(s)
tagged 'mkinitrd-6_5_2'
1 | niro | 532 | Busybox TODO |
2 | |||
3 | niro | 1123 | Harvest patches from |
4 | http://git.openembedded.org/cgit.cgi/openembedded/tree/recipes/busybox/ | ||
5 | |||
6 | niro | 532 | Stuff that needs to be done. This is organized by who plans to get around to |
7 | doing it eventually, but that doesn't mean they "own" the item. If you want to | ||
8 | do one of these bounce an email off the person it's listed under to see if they | ||
9 | have any suggestions how they plan to go about it, and to minimize conflicts | ||
10 | between your work and theirs. But otherwise, all of these are fair game. | ||
11 | |||
12 | niro | 984 | Rob Landley suggested this: |
13 | Implement bb_realpath() that can handle NULL on non-glibc. | ||
14 | niro | 532 | |
15 | sh | ||
16 | niro | 984 | The command shell situation is a mess. We have two different |
17 | niro | 532 | shells that don't really share any code, and the "standalone shell" doesn't |
18 | work all that well (especially not in a chroot environment), due to apps not | ||
19 | niro | 816 | being reentrant. |
20 | niro | 984 | |
21 | niro | 532 | Do a SUSv3 audit |
22 | Look at the full Single Unix Specification version 3 (available online at | ||
23 | "http://www.opengroup.org/onlinepubs/009695399/nfindex.html") and | ||
24 | figure out which of our apps are compliant, and what we're missing that | ||
25 | we might actually care about. | ||
26 | |||
27 | Even better would be some kind of automated compliance test harness that | ||
28 | exercises each command line option and the various corner cases. | ||
29 | niro | 984 | |
30 | niro | 532 | Internationalization |
31 | How much internationalization should we do? | ||
32 | |||
33 | The low hanging fruit is UTF-8 character set support. We should do this. | ||
34 | niro | 1123 | See TODO_unicode file. |
35 | niro | 532 | |
36 | We also have lots of hardwired english text messages. Consolidating this | ||
37 | into some kind of message table not only makes translation easier, but | ||
38 | also allows us to consolidate redundant (or close) strings. | ||
39 | |||
40 | We probably don't want to be bloated with locale support. (Not unless we | ||
41 | can cleanly export it from our underlying C library without having to | ||
42 | concern ourselves with it directly. Perhaps a few specific things like a | ||
43 | config option for "date" are low hanging fruit here?) | ||
44 | |||
45 | What level should things happen at? How much do we care about | ||
46 | internationalizing the text console when X11 and xterms are so much better | ||
47 | at it? (There's some infrastructure here we don't implement: The | ||
48 | "unicode_start" and "unicode_stop" shell scripts need "vt-is-UTF8" and a | ||
49 | --unicode option to loadkeys. That implies a real loadkeys/dumpkeys | ||
50 | implementation to replace loadkmap/dumpkmap. Plus messing with console font | ||
51 | loading. Is it worth it, or do we just say "use X"?) | ||
52 | |||
53 | Individual compilation of applets. | ||
54 | It would be nice if busybox had the option to compile to individual applets, | ||
55 | for people who want an alternate implementation less bloated than the gnu | ||
56 | utils (or simply with less political baggage), but without it being one big | ||
57 | executable. | ||
58 | |||
59 | Turning libbb into a real dll is another possibility, especially if libbb | ||
60 | could export some of the other library interfaces we've already more or less | ||
61 | got the code for (like zlib). | ||
62 | niro | 1123 | |
63 | niro | 532 | buildroot - Make a "dogfood" option |
64 | Busybox 1.1 will be capable of replacing most gnu packages for real world | ||
65 | use, such as developing software or in a live CD. It needs wider testing. | ||
66 | |||
67 | Busybox should now be able to replace bzip2, coreutils, e2fsprogs, file, | ||
68 | findutils, gawk, grep, inetutils, less, modutils, net-tools, patch, procps, | ||
69 | sed, shadow, sysklogd, sysvinit, tar, util-linux, and vim. The resulting | ||
70 | system should be self-hosting (I.E. able to rebuild itself from source | ||
71 | code). This means it would need (at least) binutils, gcc, and make, or | ||
72 | equivalents. | ||
73 | |||
74 | It would be a good "eating our own dogfood" test if buildroot had the option | ||
75 | of using a "make allyesconfig" busybox instead of the all of the above | ||
76 | packages. Anything that's wrong with the resulting system, we can fix. (It | ||
77 | would be nice to be able to upgrade busybox to be able to replace bash and | ||
78 | diffutils as well, but we're not there yet.) | ||
79 | |||
80 | One example of an existing system that does this already is Firmware Linux: | ||
81 | http://www.landley.net/code/firmware | ||
82 | niro | 1123 | |
83 | niro | 532 | initramfs |
84 | Busybox should have a sample initramfs build script. This depends on | ||
85 | bbsh, mdev, and switch_root. | ||
86 | niro | 1123 | |
87 | niro | 532 | mkdep |
88 | Write a mkdep that doesn't segfault if there's a directory it doesn't | ||
89 | have permission to read, isn't based on manually editing the output of | ||
90 | lexx and yacc, doesn't make such a mess under include/config, etc. | ||
91 | niro | 1123 | |
92 | niro | 532 | Group globals into unions of structures. |
93 | Go through and turn all the global and static variables into structures, | ||
94 | and have all those structures be in a big union shared between processes, | ||
95 | so busybox uses less bss. (This is a big win on nommu machines.) See | ||
96 | sed.c and mdev.c for examples. | ||
97 | niro | 1123 | |
98 | niro | 532 | Go through bugs.busybox.net and close out all of that somehow. |
99 | This one's open to everybody, but I'll wind up doing it... | ||
100 | |||
101 | niro | 816 | Bernhard Reutner-Fischer <busybox@busybox.net> suggests to look at these: |
102 | niro | 532 | New debug options: |
103 | -Wlarger-than-127 | ||
104 | Cleanup any big users | ||
105 | Collate BUFSIZ IOBUF_SIZE MY_BUF_SIZE PIPE_PROGRESS_SIZE BUFSIZE PIPESIZE | ||
106 | make bb_common_bufsiz1 configurable, size wise. | ||
107 | make pipesize configurable, size wise. | ||
108 | Use bb_common_bufsiz1 throughout applets! | ||
109 | |||
110 | As yet unclaimed: | ||
111 | |||
112 | ---- | ||
113 | diff | ||
114 | Make sure we handle empty files properly: | ||
115 | From the patch man page: | ||
116 | |||
117 | you can remove a file by sending out a context diff that compares | ||
118 | the file to be deleted with an empty file dated the Epoch. The | ||
119 | file will be removed unless patch is conforming to POSIX and the | ||
120 | -E or --remove-empty-files option is not given. | ||
121 | --- | ||
122 | patch | ||
123 | Should have simple fuzz factor support to apply patches at an offset which | ||
124 | shouldn't take up too much space. | ||
125 | |||
126 | And while we're at it, a new patch filename quoting format is apparently | ||
127 | coming soon: http://marc.theaimsgroup.com/?l=git&m=112927316408690&w=2 | ||
128 | --- | ||
129 | stty / catv | ||
130 | stty's visible() function and catv's guts are identical. Merge them into | ||
131 | an appropriate libbb function. | ||
132 | niro | 816 | --- |
133 | struct suffix_mult | ||
134 | Several duplicate users of: grep -r "1024\*1024" * -B2 -A1 | ||
135 | Merge to a single size_suffixes[] in libbb. | ||
136 | Users: head tail od_bloaty hexdump and (partially as it wouldn't hurt) svlogd | ||
137 | --- | ||
138 | tail | ||
139 | ./busybox tail -f foo.c~ TODO | ||
140 | should not print fmt=header_fmt for subsequent date >> TODO; i.e. only | ||
141 | fmt+ if another (not the current) file did change | ||
142 | niro | 532 | |
143 | Architectural issues: | ||
144 | |||
145 | bb_close() with fsync() | ||
146 | We should have a bb_close() in place of normal close, with a CONFIG_ option | ||
147 | to not just check the return value of close() for an error, but fsync(). | ||
148 | Close can't reliably report anything useful because if write() accepted the | ||
149 | data then it either went out to the network or it's in cache or a pipe | ||
150 | buffer. Either way, there's no guarantee it'll make it to its final | ||
151 | destination before close() gets called, so there's no guarantee that any | ||
152 | error will be reported. | ||
153 | |||
154 | You need to call fsync() if you care about errors that occur after write(), | ||
155 | but that can have a big performance impact. So make it a config option. | ||
156 | --- | ||
157 | Unify archivers | ||
158 | Lots of archivers have the same general infrastructure. The directory | ||
159 | traversal code should be factored out, and the guts of each archiver could | ||
160 | be some setup code and a series of callbacks for "add this file", | ||
161 | "add this directory", "add this symlink" and so on. | ||
162 | |||
163 | This could clean up tar and zip, and make it cheaper to add cpio and ar | ||
164 | write support, and possibly even cheaply add things like mkisofs or | ||
165 | mksquashfs someday, if they become relevant. | ||
166 | --- | ||
167 | Text buffer support. | ||
168 | Several existing applets (sort, vi, less...) read | ||
169 | niro | 984 | a whole file into memory and act on it. Use open_read_close(). |
170 | niro | 532 | --- |
171 | Memory Allocation | ||
172 | We have a CONFIG_BUFFER mechanism that lets us select whether to do memory | ||
173 | allocation on the stack or the heap. Unfortunately, we're not using it much. | ||
174 | We need to audit our memory allocations and turn a lot of malloc/free calls | ||
175 | into RESERVE_CONFIG_BUFFER/RELEASE_CONFIG_BUFFER. | ||
176 | For a start, see e.g. make EXTRA_CFLAGS=-Wlarger-than-64 | ||
177 | |||
178 | And while we're at it, many of the CONFIG_FEATURE_CLEAN_UP #ifdefs will be | ||
179 | optimized out by the compiler in the stack allocation case (since there's no | ||
180 | free for an alloca()), and this means that various cleanup loops that just | ||
181 | call free might also be optimized out by the compiler if written right, so | ||
182 | we can yank those #ifdefs too, and generally clean up the code. | ||
183 | --- | ||
184 | FEATURE_CLEAN_UP | ||
185 | This is more an unresolved issue than a to-do item. More thought is needed. | ||
186 | |||
187 | niro | 984 | Normally we rely on exit() to free memory, close files and unmap segments |
188 | niro | 532 | for us. This makes most calls to free(), close(), and unmap() optional in |
189 | busybox applets that don't intend to run for very long, and optional stuff | ||
190 | can be omitted to save size. | ||
191 | |||
192 | The idea was raised that we could simulate fork/exit with setjmp/longjmp | ||
193 | for _really_ brainless embedded systems, or speed up the standalone shell | ||
194 | by not forking. Doing so would require a reliable FEATURE_CLEAN_UP. | ||
195 | Unfortunately, this isn't as easy as it sounds. | ||
196 | |||
197 | The problem is, lots of things exit(), sometimes unexpectedly (xmalloc()) | ||
198 | and sometimes reliably (bb_perror_msg_and_die() or show_usage()). This | ||
199 | jumps out of the normal flow control and bypasses any cleanup code we | ||
200 | put at the end of our applets. | ||
201 | |||
202 | It's possible to add hooks to libbb functions like xmalloc() and xopen() | ||
203 | to add their entries to a linked list, which could be traversed and | ||
204 | freed/closed automatically. (This would need to be able to free just the | ||
205 | entries after a checkpoint to be usable for a forkless standalone shell. | ||
206 | You don't want to free the shell's own resources.) | ||
207 | |||
208 | Right now, FEATURE_CLEAN_UP is more or less a debugging aid, to make things | ||
209 | like valgrind happy. It's also documentation of _what_ we're trusting | ||
210 | exit() to clean up for us. But new infrastructure to auto-free stuff would | ||
211 | render the existing FEATURE_CLEAN_UP code redundant. | ||
212 | |||
213 | For right now, exit() handles it just fine. | ||
214 | |||
215 | |||
216 | Minor stuff: | ||
217 | watchdog.c could autodetect the timer duration via: | ||
218 | if(!ioctl (fd, WDIOC_GETTIMEOUT, &tmo)) timer_duration = 1 + (tmo / 2); | ||
219 | Unfortunately, that needs linux/watchdog.h and that contains unfiltered | ||
220 | kernel types on some distros, which breaks the build. | ||
221 | --- | ||
222 | use bb_error_msg where appropriate: See | ||
223 | egrep "(printf.*\([[:space:]]*(stderr|2)|[^_]write.*\([[:space:]]*(stderr|2))" | ||
224 | --- | ||
225 | use bb_perror_msg where appropriate: See | ||
226 | egrep "[^_]perror" | ||
227 | --- | ||
228 | possible code duplication ingroup() and is_a_group_member() | ||
229 | --- | ||
230 | niro | 984 | Move __get_hz() to a better place and (re)use it in route.c, ash.c |
231 | niro | 532 | --- |
232 | niro | 816 | See grep -r strtod |
233 | Alot of duplication that wants cleanup. | ||
234 | --- | ||
235 | in_ether duplicated in network/{interface,ifconfig}.c | ||
236 | --- | ||
237 | niro | 984 | unify progress_meter. wget, flash_eraseall, pipe_progress, fbsplash, setfiles. |
238 | --- | ||
239 | niro | 1123 | support start-stop-daemon -d <chdir-path> |
240 | niro | 532 | --- |
241 | vdprintf() -> similar sized functionality | ||
242 | --- | ||
243 | niro | 984 | |
244 | (TODO list after discussion 11.05.2009) | ||
245 | |||
246 | * shrink tc/brctl/ip | ||
247 | tc/brctl seem like fairly large things to try and tackle in your timeframe, | ||
248 | and i think people have posted attempts in the past. Adding additional | ||
249 | options to ip though seems reasonable. | ||
250 | |||
251 | * add tests for some applets | ||
252 | |||
253 | * implement POSIX utilities and audit them for POSIX conformance. then | ||
254 | audit them for GNU conformance. then document all your findings in a new | ||
255 | doc/conformance.txt file while perhaps implementing some of the missing | ||
256 | features. | ||
257 | you can find the latest POSIX documentation (1003.1-2008) here: | ||
258 | http://www.opengroup.org/onlinepubs/9699919799/ | ||
259 | and the complete list of all utilities that POSIX covers: | ||
260 | http://www.opengroup.org/onlinepubs/9699919799/idx/utilities.html | ||
261 | The first step would to generate a file/matrix what is already archived | ||
262 | (also IPV6) | ||
263 | |||
264 | * implement 'at' | ||
265 | |||
266 | * rpcbind (former portmap) or equivalent | ||
267 | so that we don't have to use -o nolock on nfs mounts | ||
268 | |||
269 | * check IPV6 compliance | ||
270 | |||
271 | * generate a mini example using kernel+busybox only (+libc) for example | ||
272 | |||
273 | * more support for advanced linux 2.6.x features, see: iotop | ||
274 | most likely there is more | ||
275 | |||
276 | * even more support for statistics: mpstat, iostat, powertop.... |