linux/Documentation/IO-mapping.txt

[ NOTE: The virt_to_bus() and bus_to_virt() functions have been
	superseded by the functionality provided by the PCI DMA
	interface (see Documentation/DMA-mapping.txt).  They continue
	to be documented below for historical purposes, but new code
	must not use them. --davidm 00/12/12 ]

[ This is a mail message in response to a query on IO mapping, thus the
  strange format for a "document" ]

The AHA-1542 is a bus-master device, and your patch makes the driver give the
controller the physical address of the buffers, which is correct on x86
(because all bus master devices see the physical memory mappings directly). 

However, on many setups, there are actually _three_ different ways of looking
at memory addresses, and in this case we actually want the third, the
so-called "bus address". 

Essentially, the three ways of addressing memory are (this is "real memory",
that is, normal RAM--see later about other details): 

 - CPU untranslated.  This is the "physical" address.  Physical address 
   0 is what the CPU sees when it drives zeroes on the memory bus.

 - CPU translated address. This is the "virtual" address, and is 
   completely internal to the CPU itself with the CPU doing the appropriate
   translations into "CPU untranslated". 

 - bus address. This is the address of memory as seen by OTHER devices, 
   not the CPU. Now, in theory there could be many different bus 
   addresses, with each device seeing memory in some device-specific way, but
   happily most hardware designers aren't actually actively trying to make
   things any more complex than necessary, so you can assume that all 
   external hardware sees the memory the same way. 

Now, on normal PCs the bus address is exactly the same as the physical
address, and things are very simple indeed. However, they are that simple
because the memory and the devices share the same address space, and that is
not generally necessarily true on other PCI/ISA setups. 

Now, just as an example, on the PReP (PowerPC Reference Platform), the 
CPU sees a memory map something like this (this is from memory):

	0-2 GB		"real memory"
	2 GB-3 GB	"system IO" (inb/out and similar accesses on x86)
	3 GB-4 GB 	"IO memory" (shared memory over the IO bus)

Now, that looks simple enough. However, when you look at the same thing from
the viewpoint of the devices, you have the reverse, and the physical memory
address 0 actually shows up as address 2 GB for any IO master.

So when the CPU wants any bus master to write to physical memory 0, it 
has to give the master address 0x80000000 as the memory address.

So, for example, depending on how the kernel is actually mapped on the 
PPC, you can end up with a setup like this:

 physical address:	0
 virtual address:	0xC0000000
 bus address:		0x80000000

where all the addresses actually point to the same thing.  It's just seen 
through different translations..

Similarly, on the Alpha, the normal translation is

 physical address:	0
 virtual address:	0xfffffc0000000000
 bus address:		0x40000000

(but there are also Alphas where the physical address and the bus address
are the same). 

Anyway, the way to look up all these translations, you do

	#include <asm/io.h>

	phys_addr = virt_to_phys(virt_addr);
	virt_addr = phys_to_virt(phys_addr);
	 bus_addr = virt_to_bus(virt_addr);
	virt_addr = bus_to_virt(bus_addr);

Now, when do you need these?

You want the _virtual_ address when you are actually going to access that 
pointer from the kernel. So you can have something like this:

	/*
	 * this is the hardware "mailbox" we use to communicate with
	 * the controller. The controller sees this directly.
	 */
	struct mailbox {
		__u32 status;
		__u32 bufstart;
		__u32 buflen;
		..
	} mbox;

		unsigned char * retbuffer;

		/* get the address from the controller */
		retbuffer = bus_to_virt(mbox.bufstart);
		switch (retbuffer[0]) {
			case STATUS_OK:
				...

on the other hand, you want the bus address when you have a buffer that 
you want to give to the controller:

	/* ask the controller to read the sense status into "sense_buffer" */
	mbox.bufstart = virt_to_bus(&sense_buffer);
	mbox.buflen = sizeof(sense_buffer);
	mbox.status = 0;
	notify_controller(&mbox);

And you generally _never_ want to use the physical address, because you can't
use that from the CPU (the CPU only uses translated virtual addresses), and
you can't use it from the bus master. 

So why do we care about the physical address at all? We do need the physical
address in some cases, it's just not very often in normal code.  The physical
address is needed if you use memory mappings, for example, because the
"remap_pfn_range()" mm function wants the physical address of the memory to
be remapped as measured in units of pages, a.k.a. the pfn (the memory
management layer doesn't know about devices outside the CPU, so it
shouldn't need to know about "bus addresses" etc).

NOTE NOTE NOTE! The above is only one part of the whole equation. The above
only talks about "real memory", that is, CPU memory (RAM). 

There is a completely different type of memory too, and that's the "shared
memory" on the PCI or ISA bus. That's generally not RAM (although in the case
of a video graphics card it can be normal DRAM that is just used for a frame
buffer), but can be things like a packet buffer in a network card etc. 

This memory is called "PCI memory" or "shared memory" or "IO memory" or
whatever, and there is only one way to access it: the readb/writeb and
related functions. You should never take the address of such memory, because
there is really nothing you can do with such an address: it's not
conceptually in the same memory space as "real memory" at all, so you cannot
just dereference a pointer. (Sadly, on x86 it _is_ in the same memory space,
so on x86 it actually works to just deference a pointer, but it's not
portable). 

For such memory, you can do things like

 - reading:
	/*
	 * read first 32 bits from ISA memory at 0xC0000, aka
	 * C000:0000 in DOS terms
	 */
	unsigned int signature = isa_readl(0xC0000);

 - remapping and writing:
	/*
	 * remap framebuffer PCI memory area at 0xFC000000,
	 * size 1MB, so that we can access it: We can directly
	 * access only the 640k-1MB area, so anything else
	 * has to be remapped.
	 */
	char * baseptr = ioremap(0xFC000000, 1024*1024);

	/* write a 'A' to the offset 10 of the area */
	writeb('A',baseptr+10);

	/* unmap when we unload the driver */
	iounmap(baseptr);

 - copying and clearing:
	/* get the 6-byte Ethernet address at ISA address E000:0040 */
	memcpy_fromio(kernel_buffer, 0xE0040, 6);
	/* write a packet to the driver */
	memcpy_toio(0xE1000, skb->data, skb->len);
	/* clear the frame buffer */
	memset_io(0xA0000, 0, 0x10000);

OK, that just about covers the basics of accessing IO portably.  Questions?
Comments? You may think that all the above is overly complex, but one day you
might find yourself with a 500 MHz Alpha in front of you, and then you'll be
happy that your driver works ;)

Note that kernel versions 2.0.x (and earlier) mistakenly called the
ioremap() function "vremap()".  ioremap() is the proper name, but I
didn't think straight when I wrote it originally.  People who have to
support both can do something like:
 
	/* support old naming silliness */
	#if LINUX_VERSION_CODE < 0x020100                                     
	#define ioremap vremap
	#define iounmap vfree                                                     
	#endif
 
at the top of their source files, and then they can use the right names
even on 2.0.x systems. 

And the above sounds worse than it really is.  Most real drivers really
don't do all that complex things (or rather: the complexity is not so
much in the actual IO accesses as in error handling and timeouts etc). 
It's generally not hard to fix drivers, and in many cases the code
actually looks better afterwards:

	unsigned long signature = *(unsigned int *) 0xC0000;
		vs
	unsigned long signature = readl(0xC0000);

I think the second version actually is more readable, no?

		Linus

1	niro	628	[ NOTE: The virt_to_bus() and bus_to_virt() functions have been
2			superseded by the functionality provided by the PCI DMA
3			interface (see Documentation/DMA-mapping.txt). They continue
4			to be documented below for historical purposes, but new code
5			must not use them. --davidm 00/12/12 ]
6
7			[ This is a mail message in response to a query on IO mapping, thus the
8			strange format for a "document" ]
9
10			The AHA-1542 is a bus-master device, and your patch makes the driver give the
11			controller the physical address of the buffers, which is correct on x86
12			(because all bus master devices see the physical memory mappings directly).
13
14			However, on many setups, there are actually _three_ different ways of looking
15			at memory addresses, and in this case we actually want the third, the
16			so-called "bus address".
17
18			Essentially, the three ways of addressing memory are (this is "real memory",
19			that is, normal RAM--see later about other details):
20
21			- CPU untranslated. This is the "physical" address. Physical address
22			0 is what the CPU sees when it drives zeroes on the memory bus.
23
24			- CPU translated address. This is the "virtual" address, and is
25			completely internal to the CPU itself with the CPU doing the appropriate
26			translations into "CPU untranslated".
27
28			- bus address. This is the address of memory as seen by OTHER devices,
29			not the CPU. Now, in theory there could be many different bus
30			addresses, with each device seeing memory in some device-specific way, but
31			happily most hardware designers aren't actually actively trying to make
32			things any more complex than necessary, so you can assume that all
33			external hardware sees the memory the same way.
34
35			Now, on normal PCs the bus address is exactly the same as the physical
36			address, and things are very simple indeed. However, they are that simple
37			because the memory and the devices share the same address space, and that is
38			not generally necessarily true on other PCI/ISA setups.
39
40			Now, just as an example, on the PReP (PowerPC Reference Platform), the
41			CPU sees a memory map something like this (this is from memory):
42
43			0-2 GB "real memory"
44			2 GB-3 GB "system IO" (inb/out and similar accesses on x86)
45			3 GB-4 GB "IO memory" (shared memory over the IO bus)
46
47			Now, that looks simple enough. However, when you look at the same thing from
48			the viewpoint of the devices, you have the reverse, and the physical memory
49			address 0 actually shows up as address 2 GB for any IO master.
50
51			So when the CPU wants any bus master to write to physical memory 0, it
52			has to give the master address 0x80000000 as the memory address.
53
54			So, for example, depending on how the kernel is actually mapped on the
55			PPC, you can end up with a setup like this:
56
57			physical address: 0
58			virtual address: 0xC0000000
59			bus address: 0x80000000
60
61			where all the addresses actually point to the same thing. It's just seen
62			through different translations..
63
64			Similarly, on the Alpha, the normal translation is
65
66			physical address: 0
67			virtual address: 0xfffffc0000000000
68			bus address: 0x40000000
69
70			(but there are also Alphas where the physical address and the bus address
71			are the same).
72
73			Anyway, the way to look up all these translations, you do
74
75			#include <asm/io.h>
76
77			phys_addr = virt_to_phys(virt_addr);
78			virt_addr = phys_to_virt(phys_addr);
79			bus_addr = virt_to_bus(virt_addr);
80			virt_addr = bus_to_virt(bus_addr);
81
82			Now, when do you need these?
83
84			You want the _virtual_ address when you are actually going to access that
85			pointer from the kernel. So you can have something like this:
86
87			/*
88			* this is the hardware "mailbox" we use to communicate with
89			* the controller. The controller sees this directly.
90			*/
91			struct mailbox {
92			__u32 status;
93			__u32 bufstart;
94			__u32 buflen;
95			..
96			} mbox;
97
98			unsigned char * retbuffer;
99
100			/* get the address from the controller */
101			retbuffer = bus_to_virt(mbox.bufstart);
102			switch (retbuffer[0]) {
103			case STATUS_OK:
104			...
105
106			on the other hand, you want the bus address when you have a buffer that
107			you want to give to the controller:
108
109			/* ask the controller to read the sense status into "sense_buffer" */
110			mbox.bufstart = virt_to_bus(&sense_buffer);
111			mbox.buflen = sizeof(sense_buffer);
112			mbox.status = 0;
113			notify_controller(&mbox);
114
115			And you generally _never_ want to use the physical address, because you can't
116			use that from the CPU (the CPU only uses translated virtual addresses), and
117			you can't use it from the bus master.
118
119			So why do we care about the physical address at all? We do need the physical
120			address in some cases, it's just not very often in normal code. The physical
121			address is needed if you use memory mappings, for example, because the
122			"remap_pfn_range()" mm function wants the physical address of the memory to
123			be remapped as measured in units of pages, a.k.a. the pfn (the memory
124			management layer doesn't know about devices outside the CPU, so it
125			shouldn't need to know about "bus addresses" etc).
126
127			NOTE NOTE NOTE! The above is only one part of the whole equation. The above
128			only talks about "real memory", that is, CPU memory (RAM).
129
130			There is a completely different type of memory too, and that's the "shared
131			memory" on the PCI or ISA bus. That's generally not RAM (although in the case
132			of a video graphics card it can be normal DRAM that is just used for a frame
133			buffer), but can be things like a packet buffer in a network card etc.
134
135			This memory is called "PCI memory" or "shared memory" or "IO memory" or
136			whatever, and there is only one way to access it: the readb/writeb and
137			related functions. You should never take the address of such memory, because
138			there is really nothing you can do with such an address: it's not
139			conceptually in the same memory space as "real memory" at all, so you cannot
140			just dereference a pointer. (Sadly, on x86 it _is_ in the same memory space,
141			so on x86 it actually works to just deference a pointer, but it's not
142			portable).
143
144			For such memory, you can do things like
145
146			- reading:
147			/*
148			* read first 32 bits from ISA memory at 0xC0000, aka
149			* C000:0000 in DOS terms
150			*/
151			unsigned int signature = isa_readl(0xC0000);
152
153			- remapping and writing:
154			/*
155			* remap framebuffer PCI memory area at 0xFC000000,
156			* size 1MB, so that we can access it: We can directly
157			* access only the 640k-1MB area, so anything else
158			* has to be remapped.
159			*/
160			char * baseptr = ioremap(0xFC000000, 1024*1024);
161
162			/* write a 'A' to the offset 10 of the area */
163			writeb('A',baseptr+10);
164
165			/* unmap when we unload the driver */
166			iounmap(baseptr);
167
168			- copying and clearing:
169			/* get the 6-byte Ethernet address at ISA address E000:0040 */
170			memcpy_fromio(kernel_buffer, 0xE0040, 6);
171			/* write a packet to the driver */
172			memcpy_toio(0xE1000, skb->data, skb->len);
173			/* clear the frame buffer */
174			memset_io(0xA0000, 0, 0x10000);
175
176			OK, that just about covers the basics of accessing IO portably. Questions?
177			Comments? You may think that all the above is overly complex, but one day you
178			might find yourself with a 500 MHz Alpha in front of you, and then you'll be
179			happy that your driver works ;)
180
181			Note that kernel versions 2.0.x (and earlier) mistakenly called the
182			ioremap() function "vremap()". ioremap() is the proper name, but I
183			didn't think straight when I wrote it originally. People who have to
184			support both can do something like:
185
186			/* support old naming silliness */
187			#if LINUX_VERSION_CODE < 0x020100
188			#define ioremap vremap
189			#define iounmap vfree
190			#endif
191
192			at the top of their source files, and then they can use the right names
193			even on 2.0.x systems.
194
195			And the above sounds worse than it really is. Most real drivers really
196			don't do all that complex things (or rather: the complexity is not so
197			much in the actual IO accesses as in error handling and timeouts etc).
198			It's generally not hard to fix drivers, and in many cases the code
199			actually looks better afterwards:
200
201			unsigned long signature = (unsigned int ) 0xC0000;
202			vs
203			unsigned long signature = readl(0xC0000);
204
205			I think the second version actually is more readable, no?
206
207			Linus
208