PowerPC Linux On The ML510

Introduction

This page contains information about using open source Linux on the ML510 board from Xilinx. In general, this page is only intended to cover specific areas of the ML510 board that are unique to it. A basic Linux kernel (no PCI) can be loaded and ran on this board in the same manner as the other Xilinx boards such as the ML507.

One of the unique features of the board is that it's designed to be a motherboard similar to a PC motherboard in that it has PCI slots.

Thanks to the work of others in the open source community, specifically Roderick Colenbrander, there were patches that made it into the mainline kernel for the ML510 board. A device tree, virtex440-ml510.dts, is in the kernel tree such that a kernel can be built easily for the ML510.

Special Notes

  1. This page only contains information about using PowerPC 440 on the ML510, not MicroBlaze. A basic MicroBlaze Linux kernel (no PCI) can be loaded and ran on the board in the same way as the other Xilinx boards. PCI work for MicroBlaze Linux is still in progress.
  1. This information assumes the user already knows how to build open source Linux based on the information for the PowerPC Linux kernel contained on the page http://xilinx.wikidot.com/powerpc-linux. This page does not duplicate information found on that page for basic kernel building, loading and running.

Hardware Design

A reference design for the ML510 with PCI, ml510_bsb1_pcores_ppc440, exists on xilinx.com, but this reference system will not work well with Linux due to address translation not being turned off on a BAR for PCI.

The following links contain a system bit file and MHS file that correspond to the virtex440-ml510.dts file in the kernel tree. This system is only intended to help establish a baseline as new systems can be built from Base System Builder (BSB) in the Xilinx EDK. Note that a BSB system needs one change to work with Linux for PCI.

download link for ML510 PowerPC bit stream

download link for ML510 PowerPC MHS file

Building A ML510 PCI System In The EDK

The following steps provide a summary of the work required to build a system. Xilinx EDK 11.5 was used for the testing that is documented.

1. Generate a system for the ML510 using Base System Builder in the EDK.
2. Before building the system, for the PCI core, change the parameter C_IPIFBAR2PCIBAR_0 to match the parameter C_IPIFBAR_0 such that there is no address translation turned on for BAR 0.
3. Build the system in the EDK.
4. Generate a device tree for the system.
5. Remove a memory from the device tree, leaving the memory that starts at address 0.
6. Add the extra information for PCI that is not generated by the device tree generator at this time.

Known Issues

Device Tree Generator / Kernel Boot

For the ML510, Base System Builder in the EDK builds a system with 2 memory DIMMs which results in 2 memories in the device tree. This will cause the kernel not to boot. Remove the memory which is not located at address 0.

5 Volt Slots

The kernel only supports the 3.3V slots on the ML510 at this time.


Kernel Configuration

The kernel should be configured 1st by using the virtex5_defconfig in the kernel as illustrated in the following command.

bash> make ARCH=powerpc 44x/virtex5_defconfig

Now PCI and ML510 support need to be added manually using the menuconfig as shown in the following command.

bash> make ARCH=powerpc menuconfig

The following screenshot shows adding PCI support which is found in the Bus options menu.

ml510pcikernelconfig.bmp

The following screenshot shows adding ML510 support which is found in the Platform support menu.

ml510kernelconfig.bmp

Testing

The primary goal of this testing was for PCI.

Ethernet NICs

The following PCI NICs have been tested with this design. Testing was only using ping. Two cards were tested in both 3.3 slots at the same time.

Manufacturer Model Rate Less Obvious Driver Details
Intel PRO/1000 GT 1000 Mbit
3M 3C980C-TXM 100 Mbit From 3COM cards, select 3c590/3c900 series
Netgear FA311 100 Mbit From EISA. VLB, PCI and on board controllers, select National Semiconductor DP8381x

Serial Cards

A standard PC type serial card with 2 serial ports (8250) has tested without success.


Device Tree Details

The Xilinx device tree generator does not contain the logic to correctly setup the PCI bus such that the user must hand edit the device tree.

Device Tree Example For PCI

The following is an excerpt from the virtex440-ml510.dts file in the kernel intended only for illlustration purposes.

        plbv46_pci_0: plbv46-pci@85e00000 {
            #size-cells = <2>;
            #address-cells = <3>;
            compatible = "xlnx,plbv46-pci-1.03.a";
            device_type = "pci";
            reg = < 0x85e00000 0x10000 >;

            /*
             * The default ML510 BSB has C_IPIFBAR2PCIBAR_0 set to
             * 0 which means that a read/write to the memory mapped
             * i/o region (which starts at 0xa0000000) for pci
             * bar 0 on the plb side translates to 0.
             * It is important to set this value to 0xa0000000, so
             * that inbound and outbound pci transactions work
             * properly including DMA.
             */
            ranges = <0x02000000 0 0xa0000000 0xa0000000 0 0x20000000>;
                  0x01000000 0 0x00000000 0xf0000000 0 0x00010000>; 

            #interrupt-cells = <1>;
            interrupt-parent = <&xps_intc_0>;
            interrupt-map-mask = <0xff00 0x0 0x0 0x7>;
            interrupt-map = <
                /* IRQ mapping for pci slots and ALI M1533
                 * periperhals. In total there are 5 interrupt
                 * lines connected to a xps_intc controller.
                 * Four of them are PCI IRQ A, B, C, D and
                 * which correspond to respectively xpx_intc
                 * 5, 4, 3 and 2.  The fifth interrupt line is
                 * connected to the south bridge and this one
                 * uses irq 1 and is active high instead of
                 * active low.
                 *
                 * The M1533 contains various peripherals
                 * including AC97 audio, a modem, USB, IDE and
                 * some power management stuff. The modem
                 * isn't connected on the ML510 and the power
                 * management core also isn't used.
                 */

                /* IDSEL 0x16 / dev=6, bus=0 / PCI slot 3 */
                0x3000 0 0 1 &xps_intc_0 3 2
                0x3000 0 0 2 &xps_intc_0 2 2
                0x3000 0 0 3 &xps_intc_0 5 2
                0x3000 0 0 4 &xps_intc_0 4 2

                /* IDSEL 0x13 / dev=3, bus=1 / PCI slot 4 */
                /*
                0x11800 0 0 1 &xps_intc_0 5 0 2
                0x11800 0 0 2 &xps_intc_0 4 0 2
                0x11800 0 0 3 &xps_intc_0 3 0 2
                0x11800 0 0 4 &xps_intc_0 2 0 2
                */

                /* According to the datasheet + schematic
                 * ABCD [FPGA] of slot 5 is mapped to DABC.
                 * Testing showed that at least A maps to B,
                 * the mapping of the other pins is a guess
                 * and for that reason the lines have been
                 * commented out.
                 */
                /* IDSEL 0x15 / dev=5, bus=0 / PCI slot 5 */
                0x2800 0 0 1 &xps_intc_0 4 2
                /*
                0x2800 0 0 2 &xps_intc_0 3 2
                0x2800 0 0 3 &xps_intc_0 2 2
                0x2800 0 0 4 &xps_intc_0 5 2
                */

                /* IDSEL 0x12 / dev=2, bus=1 / PCI slot 6 */
                /*
                0x11000 0 0 1 &xps_intc_0 4 0 2
                0x11000 0 0 2 &xps_intc_0 3 0 2
                0x11000 0 0 3 &xps_intc_0 2 0 2
                0x11000 0 0 4 &xps_intc_0 5 0 2
                */

                /* IDSEL 0x11 / dev=1, bus=0 / AC97 audio */
                0x0800 0 0 1 &i8259 7 2

                /* IDSEL 0x1b / dev=11, bus=0 / IDE */
                0x5800 0 0 1 &i8259 14 2

                /* IDSEL 0x1f / dev 15, bus=0 / 2x USB 1.1 */
                0x7800 0 0 1 &i8259 7 2
            >;
            ali_m1533 {
                #size-cells = <1>;
                #address-cells = <2>;
                i8259: interrupt-controller@20 {
                    reg = <1 0x20 2
                            1 0xa0 2
                            1 0x4d0 2>;
                    interrupt-controller;
                    device_type = "interrupt-controller";
                    #address-cells = <0>;
                    #interrupt-cells = <2>;
                    compatible = "chrp,iic";

                    /* south bridge irq is active high */
                    interrupts = <1 3>;
                    interrupt-parent = <&xps_intc_0>;
                };
            };
        } ;

PCI Binding

The following is from an email from Grant Likely that explains some details. This is intended to capture the information from the email and has not been edited for clarity.

BARs

The PCI binding is rather dense and hard to parse. The trick is in knowing how to carve it up. Fortunately, the same rules apply to a PCI ranges property as for all other ranges properties. To properly carve it up you need to know the values of #address-cells and #size-cells in the PCI bridge node, and the value of #address-cells for the parent node. Then each ranges entry can be carved up into 3 fields; the local bus address, the parent bus address, and the size.

We know that the first 3 cells specify the local bus address because #address-cells in the PCI bridge node is set to <3>. Of those 3 cells, the first cell gives the device address, and the 2nd and 3rd cells are the 64bit PCI address.
All the stuff in cell 1 is flags and device address. see section 2.2.1.1 here: http://playground.sun.com/1275/bindings/pci/pci2_1.pdf (Link broken. See: http://www.openfirmware.org/ofwg/bindings/pci/pci2_1.pdf)

There is a nice table of all the flags there. The important field is 'ss' which differentiates between config, io and memory space. The value '10' means 32 bit memory space.

Unfortunately, because of all the flags and device addresses in the first cell, it messes up translation (or in other words, the top 32 'extra' bits cause a lot of aliasing if you think of the address space as being 96 bits wide) In reality, when translating across a PCI bridge node, it is important to use the correct translator function.

In powerpc, the PCI translator simply ignores the first cell. Search for 'of_busses' in arch/powerpc/kernel/prom_parse.c.
Cells 2 and 3 are the 64 bit address, so the 0 just means address 0x0000000050000000. And we know that only one cell is used to specify the address because #address-cells of the parent node is set to <1>. #size-cells of the PCI bridge node is set to <2>, so 2 cells are used to specify the size. The 0 is just the upper 32 bits of the 64 bit size.

Interrupt Map

Interrupt map is also dense, and it took me a while to figure it out. Again, the trick is to know how to carve it up.

There are two important properties to look at for the interrupt map; interrupt-map and interrupt-map-mask. First, interrupt-map. The fields in each interrupt-map entry are: device address (#address-cells = <3>), Interrupt number (#interrupt-cells=<1>), interrupt parent (a phandle), and parent IRQ number (#interrupt-cells of parent IRQ
controller). For example, for the ml510:

interrupt-map-mask = <0xff00 0x0 0x0 0x7>;
interrupt-map = <
/* IDSEL 0x16 / dev=6, bus=0 / PCI slot 3 */
0x3000 0 0 1 &xps_intc_0 3 2
0x3000 0 0 2 &xps_intc_0 2 2
0x3000 0 0 3 &xps_intc_0 5 2
0x3000 0 0 4 &xps_intc_0 4 2

Here, the device address for all 4 entries is 0x30000 0 0. Cell 1-3 is the PCI address in exactly the same form as for the ranges property. the interrupt-map-mask is applied to the address field and the IRQ number field to mask out all the irrelevant parts (only the ddddd and fff fields matter in this case).

Cell 4 is the IRQ number (1-4). Cell 5 in the IRQ controller. Cell 6 & 7 are the irq number on the parent irq controller. We know it is 2 cells because #interrupt-cells on the parent IRQ controller is set to 2.

So, parsing the above interrupt-map gives:

Device 6 (6 « 11 = 0x3000) IRQs 1-4 mapped to xps_intc_0 irqs 3, 2, 5 & 4

/* IDSEL 0x15 / dev=5, bus=0 / PCI slot 5 */
0x2800 0 0 1 &xps_intc_0 4 2

Device 5 (5 « 11 == 0x2800) IRQ 1 mapped to xps_intc_0 irq 4

/* IDSEL 0x11 / dev=1, bus=0 / AC97 audio */
0x0800 0 0 1 &i8259 7 2

Device 1 (1 « 11 == 0x0800) IRQ 1 mapped to i8259 irq 7

/* IDSEL 0x1b / dev=11, bus=0 / IDE */
0x5800 0 0 1 &i8259 14 2

Device 11 (11 « 11 == 0x5800) IRQ 1 mapped to i8259 irq 14

/* IDSEL 0x1f / dev 15, bus=0 / 2x USB 1.1 */
0x7800 0 0 1 &i8259 7 2

Device 15 (15 « 11 == 0x7800) IRQ 1 mapped to i8259 irqs 7
The Xilinx PCI controller map device numbers to the address line - 16;
so AD17 should map to device 1. Therefore:

interrupt-map-mask = <0xff00 0x0 0x0 0x7>;
interrupt-map = <
/* IDSEL 0x11 / dev=1 */
0x0800 0 0 1 &intc_0 4 2
0x0800 0 0 2 &intc_0 3 2
0x0800 0 0 3 &intc_0 2 2 >;

Not quite an idiots guide, but see here:

http://playground.sun.com/1275/practice/imap/imap0_9d.html
(Link broken. See: http://www.openfirmware.org/1275/practice/imap/imap0_9d.pdf)

Further Reading

For a more general explanation of the format of the PCI node in the device tree file, see: http://devicetree.org/Device_Tree_Usage#PCI_Host_Bridge

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License