OpenWrt Forum Archive

Topic: OpenWrt - x86_64 EletrikR4iN

The content of this topic has been archived on 23 Apr 2018. There are no obvious gaps in this topic, but there may still be some posts missing at the end.

Hello everyone.
I would like to share with the OpenWrt Community our release, originally it was created inside a bigger project to attend a big customer with facilities everywhere,  initially the idea was routing and enterprise grade VPN at low cost, well initially…but they fell in love with Luci and other OpenWrt niceties and they wanted more, subprojects where created to achieve that and we ended up with the version presented at this post, not exactly to be honest, a lot was made by me and a couple of friends (in our free time) since the project was finished 3 months ago.

With the green light from the company we decided to publish here for everyone who wants it, the first thing you’ll notice is that it’s not only an alternative build from the official x86_64 image, it’s a totally new approach and due to original requirements it contains a lot of features that the original does not.

It replaced Cisco, Mikrotik, TP Link, Ubiquiti and others appliances for routing and even some FreeNas and Nas4free installations to deal with nas stuff, if you keep reading (I know, almost a book) you’ll understand why.
This release doesn’t have routing, storage or whatever you think as the mainline, it could be a killer router, a consistent ISCSI target, a web server or Java application servlet container, make your pick…you can install only the features you need and have the functionality available in a real small box, oriented to specific functionalities or don’t, you can have it all, routing, storage, VPN and so on…

Some highlights below:

This version brings a 4.5.0 Kernel and glibc 2.23

  • ACPI CPU FREQ and Pstates on kernel, the CPU runs cooler and saves a lot of power, the fans runs quietly and you won’t need permission to take off when turning on your appliance. A device working 24/7 without power management is like leaving the room and forgetting the lights on. Power management extends your CPU life, lowers your energy bill. It just doesn’t make sense have the CPU at full speed all the time.

  • LIO/TCM enabled on kernel, including Datera targetcli through opkg for ISCSI Management.

  • NFSv4 enabled on kernel and related applications through opkg, NFSv3 still there.

  • Freeradius 3 and a freeradius-default package which installs everything to get freeradius3 up and running without manually installing the most commonly used plugins or tweaking the configuration files just to get the service running and yes…freeradius2 is still there.

  • GCC 5.0.3, yes…this release has a taste for development and you can build almost everything from the source code, if the package you need isn’t available or you want to build your own application you can do it.  A lot of others packages with development in mind were added like kernel-dev which delivers the kernel headers and supplementary files to compile your own modules, after downloading the image… have a look around and see what’s available.

  • Buildroot-deps, now you can build OpenWrt using OpenWrt, the package buildroot-deps installs every buildroot dependencies, just fire up “opkg install buildroot-deps”, clone the trunk and you are good to “make menuconfig”.

  • Luci runs under lighttpd by default, some fixes were applied to opkg and to other applications, ensuring compatibility.

  • Samba 4 and related packages are available, samba 3.6 still there

  • Boot and Recovery: at some point we needed more flexibility on boot stuff, we wanted to use a more featured recovery/failsafe option and GRUB got in the way, draconian as ever. Please, nothing wrong with GRUB but Extlinux  has always been friendlier with our goals, so we adopted Extlinux as default boot loader and a slightly changed bg-rescue for recovery, thanks to Bodo Gianonne. Although Extlinux is the boot loader, you’ll need to install the Extlinux package to get extra features, like making boot disks.

  • VPN with strongswan heavily tested on production environments., being:
    •    Site to Site (PSK or RSA with x.509 Certificates) ;
    •    Oses native clients:
              •    IOS and OSX (Cisco IPSEC);
              •    Android (IpSec-Xauth PSK);
              •    IKEv2 EAP MsCHAPv2 (using server certificate) on Windows 7 and up.

         All scenarios with Strongswan  authenticating EAP at freeradius (OpenLDAP /AD) and getting IP addresses
         from DHCP (Sometimes DNSMASQ, sometimes ISC-DHCP). IPSec-tools worked after some patches but wasn’t
         really tested, same goes to the OpenVPN.

  • Busybox, The love and hate history around busybox keeps going, the busybox implementation in this release is minimal since busybox applets doesn’t help so much on a non-embeded system, that said, almost everything is presented in its full version and sincerely it was a pain because a lot of these packages contains shell scripts with several incompatibilities with the real programs like GNU awk, grep, sed and bash, which by the way it is the default shell (yes, a static version). I know a lot of people who don’t like bash, just me and Ubuntu does, but I’m working for a long time on the field and I’m just tired of struggling, so bash works for me and has the support for development stuff and that’s what we do. Ash is still there. We didn’t have the time to review all the shell scripts against bash, but we fixed everything that we used and if something goes wrong with any of the packages you’re used to install, check the scripts for that package, post a message on the forum or… you can do better, you can fix the issue and send to us in order to make it available for everyone. When I said “full versions” I meant everything not only the binary but complementary files, man files and so on… Yes, we’re old school and we like mandb. Still there’s a lot of functionalities which busybox is in charge, some of them is nice to keep there.

  • Kmod-tools… this is another chapter because we have these guys everywhere, first the kmodloader symlinked which at runtime doesn’t  work very well with shell scripts, more than one application which loads/unloads kernel modules in the start scripts have failed in doing so and then…we have the busybox option which sometimes doesn’t track the modules dependencies. Long story short: kmodloader is there loading the modules at start-up following the order at /etc/modules.d, we didn’t change that for several reasons: It’s part of OpenWrt nature and it is compliant with buildroot model for building kernel modules, but when you call kmod tools commands at the prompt only the full versions will hear you. Modules.dep is there keeping track of dependencies.

  • Routing, it’s OpenWrt, so no comments…except that now we have horsepower to make it fast. Very often we can see posts on the web like “Ubuntu the perfect gateway”, “Make a Centos OS your gateway/firewall”, the list and titles are endless, but why is that? Just go to a store and buy a TP-Link gear and end of story, not really, we all have a TP-Link or whatever gear already. People want more from the stuff they have 24/7 turned on. Those posts are very helpful but most people think one time or another that would be very nice to have a router web interface to make things smoother. That’s the beauty of OpenWrt because it’s definitely its business, just for completeness:  as you know not always those posts works for everyone and the nightmare takes place, people give up or just end up with something that has only very basic and limited router functionalities.

  • Transfer-tools, One thing that was a thorn in our side and I think it is for most users who are adept of the official x86_64 image: it has only 50Mb of space to “/” partition. There are some ways you can expand this limitation but none of them is simple or objective, so we decided to develop a tool to do that since we’re doing this a zillion times during development initial stages and in order to make everyone lives easier we are including it on the build as a package. The image has 1GB size, most people will think: I can live with that, before I had 50MB and now I have 1GB! Yes, you can… depending on your choices obviously, for a basic router it will be okay but if you intend something like “to infinity and beyond”  our recommendation is: Write the image to a usb stick, if you intend to keep the installation on a usb stick, pick a smaller one larger than 1GB obviously, write the image and then install transfer-tools package to transfer the installed system to the final usb stick, don’t worry since it’s an exact copy of your partitions, all configurations you made (if any) will be there on the new disk. A 10MB boot partition will be created (just like the orginal disk) and the remaining space will be used by the root partition giving you a full size disk / partition. The target disk could be a USB Stick, hard disk or a SD Card. You can create image files from your system as well and keep it as a backup ready to go, just fire up transfer_to –target=/dev/sdb or whatever drive you have available for the operation.

  • Virtualization, this was a requirement on the original project, so you have open-vm-tools 10.0.7-3227872 and vmware-tools 10.0.6.54238, make your pick…All VMware tools functionality like guest memory, network and power management are available. It was tested on esxi 5.5 and 6 , not on VMware workstation, but since it worked on the big guy, no issues are expected on the little guy. Although nowadays vmware recommends open-vm-tools in any linux guest, this vmware-tools version talks differently with the hypervisor, if you want the green check on VSphere client (or Workstation?) choose vmware-tools but if the “Third party/Independent” classification doesn’t bother you go with the open-vm-tools. Functionality is exactly the same. Both packages makes use of the same kmod-vm-tools package and please avoid any headaches, install just one or another, never both, so if opkg warns you about conflicting files listen to it, uninstall one before install another. Regarding the X modules:  It’s there but it’s not configured for obvious reasons. When creating a virtual machine choose whatever guest type you want from the Linux list, we recommend “Other Linux 64 bit”

  • Java, 2 packages are available opendjk-7-jdk and openjdk-8-jdk that means an endless list of applications, like servlet containers, media servers and so on…smaller packages with only jre are on the way.

The list is very long…virtually the image can run on any box, although our tests were made mostly in rack servers little ones like Supermicro 5018A-FTN4/5018A-TN4 and bigger ones like Cisco C220 M4 and yes your old Core2 Duo boxes from your personal graveyard are welcome. AMD processors were not tested but give it a shot, basically all support for CPU’s is kernel stuff and it’s there, AMD included.

About the image:
Definitely a lot of customizations were made in essential services and packages. The deeper changes in some of them makes it almost impossible to attempt to merge the code with OpenWrt upstream and everything we made was specific for x86_64 platform without considering any other. Almost all new packages we implemented have no business with a router device environment maybe one or another like sshpass or l7-filter but most of them will require at least extroot. We intend to keep this as a parallel project, absorbing necessary and suggested changes made in upstream but walking in another path, for how long we’ll be able to do that is undetermined.
The image has the same original openwrt configuration, network to 192.168.1.1 but without DHCP server, that said you have to  manually configure the computer which will connect to perform the configurations or you can do it directly on login console (framebuffer support in kernel will give a full resolution console). Root SSH won’t work due to root user has no password set, to connect SSH use the following credentials:

User: openwrt
Password: changeme

After the connection is made sudo everything you need or create a root password (sudo passwd) and login as root. Don’t forget to change the “openwrt” user password or delete the user if don’t intend to use it. We recommend that you create your own user and add an entry to the /etc/sudoers file using openwrt user as an example.

You’ll need internet access to reach the repository and packages, the interfaces default configuration tries to reflect the most common layout for a home router:
       •    LAN  to eth0 (bridged/br-lan – static IP Address 192.168.1.1);
       •    WAN to eth1(dhcp);
       •    WAN6 to eth1(dhcp).

Anything different from above means that you’ll have to get your hands dirty and configure manually at /etc/config/network, if you need ppp don’t worry it’s installed by default and if you don’t know how to do that, ask in the post. It can take a while but you won’t be ignored. Anyway if you don’t want to use the system as a router just configure eth0 to your local network range.

One good idea would be writing some specific tutorials, which we intend to do at this place, although a lot of people here knows a lot of things, not everyone does.
We decided not to bloat the image with pre-installed packages, since the right thing to do is install only packages that will be of use, so you want luci “opkg install luci” and go from there.
We have compiled 99% of the packages from the official feeds, so most probably you’ll find what you’re looking for at the repository, but router devices specific packages are not there.

What is EletrikR4iN?
It has nothing to do with the 3D software company, we work on a company where names are given to the teams in addition to the official names, eletrikr4in is one of them among others intentionally strange names and it was given to the team that started working at the original project.

Below the link to the images:
Openwrt-EletrikR4iN-4.5.0.img.tar.gz
The link above refers to a 1GB image, if you are on windows download Rufus to write the image to a USB stick, if you are using another OS, most probably you know what to do. If you need more space just install the transfer-tools package and go from there.
Openwrt-EletrikRaiN-4.5.0.vmdk.tar.gz
VMware disk image compatible with esxi 5.x and 6x. Just create the VM, upload the vmdk to a datastore and configure the VM to use it, the image supports any vmware disk controller and vmxnet3 out of the box and don’t forget to install one of the vmware tools package to get full hypervisor support.

I think this it! Sorry for the longest post ever and have a good one.

(Last edited by k4i on 3 May 2016, 20:22)

k4i Thank You for the work.

I have downloaded and installed EletrikR4iN and it has my curiosity.

first question - are you publishing the toolchain for rebuilding it?

Directory browsing is turned off for the directory
http://openwrt.eletrikr4in.com/4.5.0/

on your server.


I like what I see so far, but wanted to make a couple of *positive suggestions*

it does not have have vi/vim installed. I had try rummage around to determine that it uses nano. in my opinion a minimal build should also include the standard set of editors, so that files can be easily edited.  I was able to install vim but only after I could get to the package management area, which required editing config files. :-)

A wiki or set of links to publicly editable google docs would be nice too so that we can share ideas.

I observed that in the case of only having one eth card - it was hard to get started using ssh, the web interface, and package manager.

If the network config has 1 eth and a wifi card, and the you have to edit /etc/config/network which I did but to make ssh talk to my laptop I had to have a private network to my laptop, which lacked access to the package manager site.

While I did figure out a way, I was thinking of putting a script in the image - which I can't do without rebuilding it.

I am excited about trying to do more with it. If you would be willing to have a chat I do have some questions, and the post was great!

Hi marchon.
Sorry if you got a hard time to perform the basics that was not our intention and thanks for your suggestions.
The thing is:
We need prioritize end users and the image isn't doing that, mainly because the image has always been installed by the company and customer people which already has the knowledge and the right tools at hand to make everything easier, just to have an idea we have today more than 70 instances running at the customer and none of them is used for wireless, the default scenario we normally find at the customer it's 4 or more ethernet connections where at least 2 of them are wan links, so this is very different from what most of the end users have at home or small business have at office.
I think one solution would be a gateway edition of the image with web interface, dhcp and everything else required for routing installed.
But we're glad that you sorted it out, we don't even had the time for some tutorials but in the next few days we'll dedicate more time and improve the image, currently we are only 3 people giving our free time to this project and we are in different timezones right now which makes everything harder than it should be.

About the toolchain:
Big companies have a lot of constraints but we're working on that, they said that it will depend on the adoption but last week some pre-sales team reached out our manager and said that they will include the "little guy" (that's how they call it) in a negotiation with some insurance company, so this is good because we can get more people involved but for now they don't want to talk about a public toolchain, they said to put this idea on hold...but sooner and I hope not later it will happen.

About the wiki:
We want to use the url in my first post to publish tutorials and talk about the image, but for now I think we can use the forum.
Your suggestions were noted and let's talk about others you may have.

Thanks once again.

(Last edited by k4i on 8 Jun 2016, 03:40)

I don't have it all quite sorted out yet.  if you would be willing to message me directly marchon (at) gmail.com we could exchange some ideas and information. 



G.

marchon wrote:

I don't have it all quite sorted out yet.  if you would be willing to message me directly marchon (at) gmail.com we could exchange some ideas and information. 



G.

I just forgot, enabled directory listing on domain.
You can reach me at kai at eletrikr4in.com

Do you have the source code available for your freeradius3?

linuxman5 wrote:

Do you have the source code available for your freeradius3?

AFAIK freeradius is opensource right?
http://freeradius.org/getting.html

k4i wrote:

About the toolchain:
Big companies have a lot of constraints but we're working on that, they said that it will depend on the adoption but last week some pre-sales team reached out our manager and said that they will include the "little guy" (that's how they call it) in a negotiation with some insurance company, so this is good because we can get more people involved but for now they don't want to talk about a public toolchain, they said to put this idea on hold...but sooner and I hope not later it will happen.

Any progress in open the toolchain to the public...

best regards eh

Wow, sounds impressive. I'd love to try it but I'd need to be able to build my own image as I already run a substantial number of customizations on my x86_64 build.

eliashaisch wrote:
k4i wrote:

About the toolchain:
Big companies have a lot of constraints but we're working on that, they said that it will depend on the adoption but last week some pre-sales team reached out our manager and said that they will include the "little guy" (that's how they call it) in a negotiation with some insurance company, so this is good because we can get more people involved but for now they don't want to talk about a public toolchain, they said to put this idea on hold...but sooner and I hope not later it will happen.

Any progress in open the toolchain to the public...

best regards eh

Notwithstanding that a binary has been (kindly) released, I'd venture to suggest that it does impose an obligation to release the toolchain and other (modified) package sources due to GPL licensing requirements.

That said, if it becomes community maintained it would ultimately be of big benefit to your customer since the ongoing cost of maintaining such a distribution can be quite high in maintenance terms. I think there is definitely a place for a version of OpenWrt focused on "bigger" systems that have more resources available than a typical consumer router.

I run a pretty heavily modified x86_64 OpenWrt system myself (although not as heavily modded as yours) and the amount of time it's taken to get it to this point is substantial.

Hi, would love to try your vm of this, but i keep getting it crash ( well kernel panic (flashing, caps lock)) what ever combination i seem to try... 
any ideas how to troubleshoot it.. (very limited knowledge in this area, but willing to learn)
my esxi 6 server is running dual xeon E5-2609's

Tried it now on two different esxi 6 servers. HP dl380 g6 and a dell precision t5600
and it falls over exactly the same time..
last line on the console says
[   3.441531 Freeing unused kernel memory: 892k (fffffffff81c32000 - ffffffff81d11000)


any ideas?

The discussion might have continued from here.