Welcome! Log In Create A New Profile


UIO driver defining more pci_alloc_consistent regions via sysfs

Posted by scott_wilson46 
I am not sure if this is the right place or the right way of doing this. Essentially I'm trying to create a device driver for an PCIe based FPGA using UIO. I want to create a bunch of buffers for the FPGA to get data back to the host (more buffers than you can allocate via the standard method using uio_info). I thought one way to do this would be to create a custom sysfs file listing the internal_addr and physical_addresses which I could then parse and use mmap to give the host access to the memory region.

So first, I do:

void __iomem *internal_addrs[8] = {0,0,0,0,0,0,0,0};
phys_addr_t phys_addrs[8] = {0,0,0,0,0,0,0,0};

static ssize_t show_mem_region(struct device *dev, struct device_attribute *attr, char *buf) {
return snprintf(buf, PAGE_SIZE, "%p %p\n%p %p\n%p %p\n%p %p\n%p %p\n%p %p\n%p %p\n%p %p\n",
(void *)internal_addrs[0],(void *)phys_addrs[0],
(void *)internal_addrs[1],(void *)phys_addrs[1],
(void *)internal_addrs[2],(void *)phys_addrs[2],
(void *)internal_addrs[3],(void *)phys_addrs[3],
(void *)internal_addrs[4],(void *)phys_addrs[4],
(void *)internal_addrs[5],(void *)phys_addrs[5],
(void *)internal_addrs[6],(void *)phys_addrs[6],
(void *)internal_addrs[7],(void *)phys_addrs[7]);


static DEVICE_ATTR(mem_region, S_IRUSR, show_mem_region, NULL);

and then in probe I do:

for (i=0; i<8; i++) {
internal_addrs = pci_alloc_consistent(pdev,
printk(KERN_INFO "Created %d byte DMA region at %p (physical %p)",
(void *)internal_addrs,
(void *)phys_addrs);

err = device_create_file(&pdev->dev, &dev_attr_mem_region);
if (err < 0)
goto out_unmap;

This gives me (/sys/class/uio/uio0/device/mem_region)

ffff88006bc00000 000000006bc00000
ffff88006b800000 000000006b800000
ffff88006b400000 000000006b400000
ffff88006b000000 000000006b000000
ffff88006ac00000 000000006ac00000
ffff88006a800000 000000006a800000
ffff88006a400000 000000006a400000
ffff88006a000000 000000006a000000

I then in my userspace code want to mmap these regions so that I can read back what the FPGA has dumped there (the FPGA gets given the physical addresses).

uint64_t * my_addr;
my_addr = (uint64_t) 0xffff88006bc00000;

fd = open("/dev/uio0", O_RDWR|O_SYNC);
map = mmap(my_addr, 4194304 , PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);

This doesn't seem to work, I always end up with:
EINVAL 22 /* Invalid argument */

I'm sure I'm doing something pretty dumb, but not sure what it is.

Out of interest I've solved this by the way. I'll try and describe what I did in case its useful for others:

The issue is (which took me a while to realize) is that calling mmap in userspace calls the mmap function in the kernel driver. Before it does that, I guess it checks to see if the offset makes sense (to see if its outside of the number of memory regions defined or something).

What I did in the end was create a simple character device driver that sits alongside the uio driver and basically contains an mmap function:

In the probe function, I setup the chardev using mmap_chardev_setup() and also add a few more things (mainly a file_operations struct and the mmap function):

static struct file_operations my_fops = {
.open = dma_open,
.release = dma_close,
.mmap = dma_mmap,
.unlocked_ioctl = dma_ioctl

static int mmap_chardev_setup() {
dev_t dev;
int r;
dev = 0;

printk(KERN_WARNING "In mmap_chardev_setup\n"winking smiley;

r = alloc_chrdev_region(&dev, 0, 1, CHARDEV_NAME);
if (r<0) {
printk(KERN_WARNING "alloc_chrdev_region failed\n"winking smiley;
return r;

devno = dev;

cdev_init(&cdev, &my_fops);
cdev.owner = THIS_MODULE;
cdev.ops = &my_fops;
r=cdev_add(&cdev, dev, 1);
if (r<0) {
printk("cdev_add failed\n"winking smiley;

cls = class_create(THIS_MODULE, CHARDEV_NAME);
if (cls == NULL) {
return -1;

device_create(cls, NULL, dev, NULL, CHARDEV_NAME);
return 0;

int dma_mmap(struct file *filep, struct vm_area_struct *vma) {
int retval;
printk(KERN_INFO "In dma_mmap, offset %lu\n", vma->vm_pgoff);

// DMA mapping!
printk(KERN_WARNING "Mapping DMA region!"winking smiley;

retval = remap_pfn_range(vma,
phys_addrs[vma->vm_pgoff] >> PAGE_SHIFT,
vma->vm_end - vma->vm_start,

if (retval)
printk(KERN_WARNING "Call to remap_pfn_range failed: %d", retval);

// Write the physical address into the buffer
*(uint64_t *)internal_addrs[vma->vm_pgoff] = phys_addrs[vma->vm_pgoff];

return retval;

This allows me to get access to the extra buffers I've created in userspace.

Your Email:


Spam prevention:
Please, solve the mathematical question and enter the answer in the input field below. This is for blocking bots that try to post this form automatically.
Question: how much is 14 plus 16?