OpenWrt Forum Archive

Topic: Tile/Tilera

The content of this topic has been archived between 15 Apr 2018 and 22 Apr 2018. There are no obvious gaps in this topic, but there may still be some posts missing at the end.

Nice works Spooky

Works on CCR1036-8G-2S+ with no Issues all ports show up and can pull a ip but the 10G ports. will boot on a CCR1009-8g-1s-1s+ but you cant use the ether1-4 ports guessing because of the switch chip they have in there for those ports most likely but the other ports work fine.  Will boot on a CCR1036-12g-4s but none of the ports show up I can connect a usb network card and that will work.

All 3 images on all devices will boot for me. I did try a hub but cant get it to work so I can do usb and usb network card on the one that has no networking. I don't know if this is my hub being bad or a power issue on the port not supplying the current needed to run a usb stick and a usb network interface.

using these commands i can mount the ubifs drives.

mkdir /mnt/RouterOS
mkdir /mnt/Kernel
mount -t ubifs ubi0:Kernel /mnt/Kernel/
mount -t ubifs ubi1:RouterOS /mnt/RouterOS/


I found for the nand image it boots and looks for a ubifs drive with the default name of ubi1:RouterOS. at first I reformatted the drive and renamed it and ran into issues because the names did not match.

I have not tested the micro-sd card slot did not have a card around at the time.

I was able to set up the usb stick to boot up with a ip and routes to make it accessible via ssh on a reboot by editing the /etc/rc file and adding in the needed commands to start and set things up on boot.

Just need to get the thing booting the kernal from internal storage and we can get rid of the need of a tftp server.

Hey Guys,

I guess I figured out how to bring up the 10G interfaces to be up, I just need someone who helps me to make the whole kernel because based on thomasthomas's description, I was unable to make it and place the output to the console, however I saw the connection between my 10G switch came up.

Many thanks.

Hi Spooky

Thanks for sharing your work with us.

We have a CCR1036-12g-4s and we report the same as HoBoPanda said: it boots fine, but no network interfaces show up at all. Is there anything that can be done about that? We could provide remote access to our machine if it helps in any way.

Also, we'd love to have tun.ko so we can test VPN performance in this device. If you have it, would you please send it to us?

We tried compiling kernel 3.17 and it boots fine, but just after we do a "exec /bin/sh" in our init script, the serial port seems not to receive any characters anymore, so we can't control de machine over the serial port. Any tips?

Thanks in advance.

loremipsum wrote:

Hi Spooky

Thanks for sharing your work with us.

We have a CCR1036-12g-4s and we report the same as HoBoPanda said: it boots fine, but no network interfaces show up at all. Is there anything that can be done about that? We could provide remote access to our machine if it helps in any way.

Also, we'd love to have tun.ko so we can test VPN performance in this device. If you have it, would you please send it to us?

We tried compiling kernel 3.17 and it boots fine, but just after we do a "exec /bin/sh" in our init script, the serial port seems not to receive any characters anymore, so we can't control de machine over the serial port. Any tips?

Thanks in advance.

Hey,

We may work together to fix this issue. I know how we can build the driver in the kernel, we have issue with puting everything to the console.

My email address is gbirincsik@gmail.com send me an email and we can work together.

Hi Loremipsum

CCR1036-12g-4S is the on-board interface driver is not applicable.
To use other models, or please use the USB-LAN.
It can be used if the chip set of ASIX AX88772.
AX88179 will not be able to use the kernel panic occurs.
Even if you build your own kernel, it is possible to ssh connected via USB-LAN.

(Last edited by spooky_boogie on 16 May 2016, 07:29)

spooky_boogie wrote:

Hi Loremipsum

CCR1036-12g-4S is the on-board interface driver is not applicable.
To use other models, or please use the USB-LAN.
It can be used if the chip set of ASIX AX88772.
AX88179 will not be able to use the kernel panic occurs.
Even if you build your own kernel, it is possible to ssh connected via USB-LAN.


I assume that is possible to resolve it anyway. My problem I am unable to put the output to the console may someone help me with that?

I used the step by step instruction what was shared here, but no output on the console.

I have CCR1036-8G-2s+

Hello.

How to use the console in the kernel that you create.

/etc/inittab
hvc0::respawn:/sbin/getty -L 115200 console
console::respawn:/sbin/getty -L 115200 ttyS0

Hello,

is there an update to this?

Nice Spooky.
maby hack working fine.

-bash-4.3$ cat /proc/meminfo
MemTotal:        2003200 kB
MemFree:         1775296 kB
Buffers:            9408 kB
Cached:            74240 kB
SwapCached:            0 kB
Active:            57216 kB
Inactive:          49472 kB
Active(anon):      23680 kB
Inactive(anon):       64 kB
Active(file):      33536 kB
Inactive(file):    49408 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                64 kB
Writeback:             0 kB
AnonPages:         23488 kB
Mapped:            17728 kB
Shmem:               576 kB
Slab:              64448 kB
SReclaimable:       6784 kB
SUnreclaim:        57664 kB
KernelStack:        6848 kB
PageTables:         8128 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     1001600 kB
Committed_AS:      30528 kB
VmallocTotal:    4194304 kB
VmallocUsed:      216576 kB
VmallocChunk:    3964864 kB
Sequestered:        5312 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:      16384 kB

-bash-4.3$ cat /proc/cpuinfo
cpu count       : 9
cpu list        : 0-8
model name      : tilegx
flags           :
cpu MHz         : 1000.000000
bogomips        : 2000.00

processor       : 0

processor       : 1

processor       : 2

processor       : 3

processor       : 4

processor       : 5

processor       : 6

processor       : 7

processor       : 8
-bash-4.3$

now get Unixbench & Runing on CCR1009

bash-4.3# ./Run -c 9 dhry2reg
make all
make[1]: Entering directory '/home/toor/temp/UnixBench'
Checking distribution of files
./pgms  exists
./src  exists
./testdir  exists
./tmp  exists
./results  exists
make[1]: Leaving directory '/home/toor/temp/UnixBench'
sh: 3dinfo: command not found

   #    #  #    #  #  #    #          #####   ######  #    #   ####   #    #
   #    #  ##   #  #   #  #           #    #  #       ##   #  #    #  #    #
   #    #  # #  #  #    ##            #####   #####   # #  #  #       ######
   #    #  #  # #  #    ##            #    #  #       #  # #  #       #    #
   #    #  #   ##  #   #  #           #    #  #       #   ##  #    #  #    #
    ####   #    #  #  #    #          #####   ######  #    #   ####   #    #

   Version 5.1.3                      Based on the Byte Magazine Unix Benchmark

   Multi-CPU version                  Version 5 revisions by Ian Smith,
                                      Sunnyvale, CA, USA
   January 13, 2011                   johantheghost at yahoo period com

Wide character in print at ./Run line 1511.
Wide character in printf at ./Run line 1542.
Use of uninitialized value in printf at ./Run line 1380.
Use of uninitialized value in printf at ./Run line 1590.

9 x Dhrystone 2 using register variables  1 2 3 4 5 6 7 8 9 10
Wide character in printf at ./Run line 1484.

========================================================================
   BYTE UNIX Benchmarks (Version 5.1.3)

   System: MikroTik: GNU/Linux
   OS: GNU/Linux -- 3.3.5 -- #4 SMP Fri Jan 27 08:53:53 UTC 2017
   Machine: tile (unknown)
   Language: en_US.euc (charmap="EUC", collate="EUC")
   CPU 0: tilegx (2000.0 bogomips)

   16:21:09 up  1:55,  2 users,  load average: 2.11, 2.34, 2.18; runlevel unknown

------------------------------------------------------------------------
Benchmark Run: Tue Jan 31 2017 16:21:09 - 16:23:21
1 CPU in system; running 9 parallel copies of tests

Dhrystone 2 using register variables       24387302.9 lps   (10.0 s, 7 samples)

System Benchmarks Partial Index              BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   24387302.9   2089.7
                                                                   ========
System Benchmarks Index Score (Partial Only)                         2089.7

bash-4.3#

Hello @fgrep, nice work the hvfs.py, it helped me a lot understanding the filesystem structure.
I made the following (not the most clean code, but it seems to do the job): http://pastebin.com/bgKuNq6U

#!/usr/bin/python

import sys, os, zlib, io
from struct import pack, unpack

if (len(sys.argv) > 1):
    hvfsdir  = sys.argv[1]
else:
    print "Use: hvfsmaker.py directory"
    sys.exit(0)

f = open("hvfs.fs", "w+b")

# Start creating our file
f.write("HvFs")

numfiles = 0
totsize = 0
totlen = 0

files = []

banner = "Tilera Hypervisor file system by ME!"
totlen += (4 - (len(banner)%4)) + len(banner)

for name in sorted(os.listdir(hvfsdir)):
    long = os.path.join(hvfsdir, name)
    filerealsize = os.stat(long).st_size
    filesize_align = (4 - (filerealsize%4)) + filerealsize
    namelen = (4 - (len(name)%4)) + len(name)
    numfiles += 1
    totsize += filerealsize
    totlen += namelen
    files.append((name,namelen,filesize_align,long,filerealsize))

# (4_HVFS + 16_HEADER + ( 16_INODE * FILES))
namebase = 4 + 16 + (16 * numfiles)

# (NAMEBASE + lenght of all filenames, keeping 1 char empty per name)
filebase = namebase + totlen

print "Files to add %d" % numfiles
print "Size to add %d" % totsize
print "Starting names at %d" % namebase
print "Starting files at %d" % filebase

# Leave header empty for now
f.seek(16,1)

# Inserting banner, it's the first entry at namebase
currentpos = f.tell()
f.seek(namebase,0)
f.write(banner)
namebase += len(banner)+1
f.seek(currentpos,0)

print "Writing inodes and files"
for file in files:
    print "Injecting file %s:name at %d and file at %d" % (file[0], namebase, filebase)
    # POS without 4_HVFS ident
    f.write(pack('<IIII', (namebase-4), (filebase-4), file[4], 0))
    currentpos = f.tell()

    # Put filename in the right place (empty trailer)
    f.seek(namebase,0)
    f.write(file[0])

    # Copy data from our file into HVFS
    f.seek(filebase,0)
    copy = file[4]
    f2 = open(file[3],'rb')
    while copy:
        block = min(1024,copy)
        data = f2.read(block)
        f.write(data)
        copy -= block
    f2.close()

    # Go back for the next file / inode
    f.seek(currentpos,0)
    namebase += file[1]
    filebase += file[2]

f.seek(0, io.SEEK_END)
fsend = f.tell()
print "done with inodes and files at %d" % fsend

# FS CRC might need to shift, create empty space if necessary (nasty hack)
if (fsend-4)%4 != 0 :
    fsend += (4 - (fsend-4)%4)
    f.seek(fsend,0)
    f.write(pack("<I", 0))

# Calculate and write header CRC (pos 5-20, we left this space empty)
f.seek(4,0)
headercrc = zlib.crc32(pack('<ccccIII',"H", "v", "F", "s", numfiles, (fsend-4), (16 + (16 * numfiles)))) & 0xffffffff
f.write(pack('<IIII', numfiles, (fsend-4), (16 + (16 * numfiles)), headercrc))

# Filesystem (skip 4_HVFS + 16_HEADER) (end of file)
f.seek(20,0)
fscrc = zlib.crc32(f.read(fsend-20)) & 0xffffffff
f.seek(fsend,0)
f.write(pack("<I", fscrc))

print "Header CRC: %X" % headercrc
print "Filesystem CRC: %X" % fscrc

f.close()

It's now more easy to create bootable code (based on the ELF included in @spooky_boogie vmlinux_initramfs, again not very clean but very functional  http://pastebin.com/u91g81zX ):

#!/bin/bash
FILE="./vmlinux_initramfs"

poself=`LANG=C grep -b -a -o -P '\x7f\x45\x4c\x46\x02\x01\x01' $FILE | cut -d ":" -f 1`
poshvfs=`LANG=C grep -b -a -o -P '\x48\x76\x46\x73' $FILE | cut -d ":" -f 1`
dd bs=1 skip=$poself count=$(($poshvfs-$poself)) < $FILE >vmlinux

cp linux-4.10.1/vmlinux.bz2 hvfs-dir/vmlinux
./hvfsmaker.py hvfs-dir/

cat hvfs.fs >>vmlinux

headersec=`readelf -a ./vmlinux | grep "Start of section headers" | awk '{print $5}'`
cursize=`du -b vmlinux | awk '{print $1}'`
dd bs=1 count=$(($headersec-$cursize-200)) </dev/zero >> vmlinux

numsec=`readelf -a ./vmlinux | grep "Number of section headers" | awk '{print $5}'`
sizesec=`readelf -a ./vmlinux | grep "Size of section headers" | awk '{print $5}'`
poselftrailer=`LANG=C grep -b -a -o -P '\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0B\x00\x00\x00\x01\x00\x00\x00' $FILE | cut -d ":" -f 1`

dd bs=1 skip=$((poselftrailer-200)) count=$((($numsec * $sizesec)+200)) < $FILE >>vmlinux

The result should be the vmlinux file that is network bootable for the CCR (tested it with CCR1036 and CCR1072), no need anymore for the manual hvfs checksum fix.
I will try to include bigger initramfs images in the coming weeks to get a test environment (my lab contains 2xCCR1036 and 1xCCR1072 for now).

(Last edited by thomasthomas on 6 Mar 2017, 02:27)

Right now I am testing on a CCR1036-8G-2S+. Spooky's kernel works for me and I am able to use the ethernet ports.

I was able to disassemble and figure out what Spooky's kernel did for the most part.
I am able to read the PHY ID on the first four ports now and I am able to get the link status. Newer kernel sources include a driver for the PHY, I am able to attach the PHY to the interface and it seems to read/set the link status properly. I am still working on figuring out how to get mPIPE sending properly. Right now the interface comes up but the PHY driver complains about the SGMII link not being up. From what I am reading, that is due to the MAC registers on the mPIPE side not being configured properly.

The built in functions to enumerate the interfaces via the hypervisor work. The code needs patched to call gxio_mpipe_init before enumeration. This still does not allow the driver to work. Code needs to be implemented to add an mdio interface for the PHY. I will be releasing my code shortly for this. It still needs more work and is very rough. The PHY/MAC index returned by the gxio_mpipe_link_open function works for the first four interfaces but after that it does not seem to work for accessing the PHYs. I was able to brute force the index/address of the PHY device but I have no way of matching each PHY to each mPIPE interface. Spooky's kernel seemed to have a lookup table of sorts that matched board name to a list of interfaces and a list of PHY IDs.

EDIT: I just finished up working on the PHY code and we actually have a link and packets moving! The SFP+ interface is still not working but it will just need a PHY driver written for it. They use the AMCC QT2025 PHY for the SFP+ interfaces. I will be releasing my patches for the kernel soon. I am currently building kernel 4.10.1 and I have ported the important patches over from the mikrotik 3.3.5 kernel. Looking to see if I can get MicroSD working shortly and then maybe even the LCD big_smile

(Last edited by josiah2009 on 6 Mar 2017, 06:57)

I made a more convenient building environment, this should help people to try this at home.

Small modified version of npk packet extractor ( https://pastebin.com/pKDjvrDa ) , header is dynamic (references are different between platforms and routeros versions). We do not care about any other file besides the kernel. Not going to spend more time on this:

$:/usr/src/ccrdev/keep# ../tools/dumpnpkkernel.py routeros-tile-6.38.3.npk
Dsize: 50
Kernel files in package:
-> boot/kernel1
$:/usr/src/ccrdev/keep#

Remember, this is the outer kernel, including the Tilera HV and Mikrotik software.

Within the /usr/src/ccrdev/src I have some important directories & files:
1) initramfs/, cross-compiled busybox including some devices, init scripts, ...

2) hvfs-dir/, used during the building of our image, contains 3 important files.
- config : you are able to modify some of the HV configuration, like exposed devices. It seems to be loaded, but not sure on all the possible options. ( https://pastebin.com/eJUSJ4iR )
- vmlinux : the inner kernel (bzip2'ed)
- initramfs.cpio.gz : cpio version of our initramfs

3) kernel sources & tilegx toolset ( http://www.mellanox.com/repository/solu … 64.tar.bz2 ), export.source file (to be loaded 'source /usr/src/ccrdev/src/export.source' before compiling your kernel: https://pastebin.com/535Z6jJz )

If your kernel is ready (vmlinux) and your initramfs is populated, use this script to build (build.sh, just keep it on the /usr/src/ccrdev/src/ : https://pastebin.com/EmL575J0 )

I have made the following scripts to make hvfs and to fix the final elf file (size, section headers, ...). This way we are not limited to the mikrotik outer kernel size (I even have images of 25mb booting fine)

/usr/src/ccrdev/tools/hvfsmaker.py : https://pastebin.com/Yz9RkXuw

/usr/src/ccrdev/tools/elf_shmove.py : https://pastebin.com/N7PP78zU

/usr/src/ccrdev/tools/elf_shput.py : https://pastebin.com/rMNFmfkv


The build script will do most of the logic (copy & bzip the kernel, making the initramfs file, populate the hvfs-dir and include it into the image), as result you should have 1 file: /usr/src/ccrdev/src/vmlinux that can be used for tftp boot on your CCR.

Just for fun we will extract some more stuff (if you only are building custom kernels you can skip these steps):

1) once to have the filesystem (/lib/modules/3.3.5)

$:/usr/src/ccrdev/keep# binwalk -e routeros-tile-6.38.3.npk

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
65536         0x10000         Squashfs filesystem, little endian, version 4.0, compression:xz, size: 12089735 bytes, 1063 inodes, blocksize: 262144 bytes, created: 2017-02-07 10:39:08
12157063      0xB98087        Executable script, shebang: "/bin/bash"
...
16395851      0xFA2E4B        Unix path: /var/pdb/system/crcbin

$:/usr/src/ccrdev/keep# ls _routeros-tile-6.38.3.npk.extracted/squashfs-root/lib/modules/3.3.5/
drivers  kernel  misc  modules.dep.system  net
$:/usr/src/ccrdev/keep#

2) once to have the inner kernel (this one is doing the work: https://raw.githubusercontent.com/torva … ct-vmlinux ). You can use it like any other custom compiled kernel in the later scripts to test all the building steps.

$:/usr/src/ccrdev/keep# binwalk boot/kernel1  | grep -A 1 bzip2
723238        0xB0926         bzip2 compressed data, block size = 900k
2807494       0x2AD6C6        xz compressed data
$:/usr/src/ccrdev/keep#
$:/usr/src/ccrdev/keep# dd if=boot/kernel1 bs=1 skip=723238 count=$((2807494-723238)) > step1
2084256+0 records in
2084256+0 records out
2084256 bytes (2.1 MB, 2.0 MiB) copied, 2.63633 s, 791 kB/s
$:/usr/src/ccrdev/keep# ../tools/extract-vmlinux step1 > mtvmlinux
$:/usr/src/ccrdev/keep# rm step1
$:/usr/src/ccrdev/keep#

I also tried to get networking up based on MT's patches and a linux 3.3.5 kernel(+ the /lib/modules, extracted out of the NPK):

https://github.com/wsxarcher/routeros-linux-patch

First it looked promising (booting, almost all modules can be loaded), but:

The patch is incomplete:
1) Missing a function called "per_cpu_port_ifindex", it should be included within the kernel (mt's inner kernel has traces of it). Required by the net/tilegx.ko

I have made the following modifications to the ./arch/tile/rb/setup.c

void per_cpu_port_ifindex(void) {
        printk("per_cpu_port_ifindex\n");
}
EXPORT_SYMBOL(per_cpu_port_ifindex);

This will allow the kernel modules to be all loaded, the function is never called (don't see any trace of the printk in the dmesg).

insmod /lib/modules/3.3.5/kernel/net/netfilter/nf_conntrack.ko
insmod /lib/modules/3.3.5/drivers/net/packet_hook.ko
insmod /lib/modules/3.3.5/drivers/net/phy_helper.ko
insmod /lib/modules/3.3.5/drivers/net/phy_amcc_qt2025.ko
insmod /lib/modules/3.3.5/drivers/net/phy_ti_tlk10232.ko
insmod /lib/modules/3.3.5/drivers/net/switch.ko
insmod /lib/modules/3.3.5/drivers/net/tilegx.ko

/ # lsmod
tilegx 75830 0 - Live 0xfffffff7101b0000
switch 44765 1 tilegx, Live 0xfffffff710170000
phy_ti_tlk10232 6729 1 tilegx, Live 0xfffffff710130000
phy_amcc_qt2025 36145 1 tilegx, Live 0xfffffff7100f0000
phy_helper 24575 4 tilegx,switch,phy_ti_tlk10232,phy_amcc_qt2025, Live 0xfffffff7100b0000
packet_hook 175073 2 tilegx,switch, Live 0xfffffff710050000
nf_conntrack 71658 1 packet_hook, Live 0xfffffff710000000
/ #

2) Lots of interfaces now, getting -1161 error (status up/down works, no packets are being forwarded). Something is still different compared to the mt's inner kernel (not just kernel module stuff)

[ 1635.501980] tile mpipe0: clsq_hwm:1023 num_blocks:1536 cutthrough:80
[ 1635.501988] tile mpipe0: hw_tx_queue_count:24
[ 1635.501995] tile mpipe0: notif_ring_count:256
[ 1635.502002] tile mpipe0: notif_group_count:32
[ 1635.502009] tile mpipe0: bucket_count:4160
[ 1635.502016] tile mpipe0: 1g devs: 8, 10g devs: 2
......
[ 1658.992279] eth9: open gbe
[ 1658.992288] eth9: do open phy:1
[ 1658.992316] eth9: equeue_size:8192 per_cpu_tx_queue_size:227 (max:8172)
[ 1658.992564] tilegx: gxio_mpipe_rules_commit failed: -1161
[ 1660.752476] eth9: phy link up (1000/full)

Long list: https://pastebin.com/Ljkas4CX

@spooky_boogie
Hi, are you still involved with porting linux to this target?

Hi everyone, Do any of you have instructions on building the kernel image to try out on a CCR1036 or any pre-build binaries (Spooky's kernel?).   thanks, Derek

Hi All,
It is original RPM package.
please try it.
150.95.184.119/RPMS/

spooky_boogie ,

thx rep.
plz send email haha .
mhoohm(g.y)

Hi,

is there any progress supporting these routers?
Does anyone want to sell a device for hacking purpose?

best regards
Waldemar

The discussion might have continued from here.