After multiple years of using OpenWRT, I built trust to this great product. That was the reason for me to use it also in AWS environment.
For now I have running OpenWrt 19.07.5 r11257-5090152ae3 as AWS EC2 t2.micro instance, all is OK.
I would like to migrate from t.2 to t3.micro instance, but during execution of the VM, it freeze on message "Waiting for root device /dev/xvda2..."
The difference between t2 and t3 is, that t3 does use Amazon EFA (Elastic Fabric Adapter) and ENA(Elastic Network Adapter), which I think is the problem during boot to connect to the xvda2 device.
That says Linux is waiting for root device which is Xen volume but can't find it. You should check with initramfs image (connect to it with serial console if it's provided by AWS) what's populated in /dev and modify GRUB config file to point root= to proper device. That should fix it. Good luck debugging.
For testing/development you can use free tier (t2.micro or t3.micro) for one year, which is at no cost.
For production you can use multiple cost optimization programs which brings significant cost savings.
I am running academic project related to security research with 50 honeynets/honeypots worldwide. Openwrt is very valuable here not only from its capabilities but also from its very low footprint in terms of storage (AWS low limit for system drive is 1GB which is for no additional costs and OpenWrt fits it very well). Also its requirements for RAM and CPU is very cost effective.
By migrating to t3.nano I will spare a lot of funds (decreasing cost by ~50%)
I tried to point root= from /dev/xd.. to partition UUID produced the same error.
In general I think that the problem is in the storage/network (EFA/ENA) driver. EC2 T2 instances use EBS-only as storage, not local storage. Migrating to any other T2 instance profile works without any problem, there s no EFA/ENA in use. T3 instances does use EFA/ENA for storage/network optimization and require to install appropriate drivers and when I migrate to T3 profile, mentioned error is returned.
I will try to play with debugging, but I am not yet sure what are the options in AWS environment.
Anyway thank you very much for your hints.
That's high science if not alchemy for me, so I rather not to try it and let it for experts. I was hoping for installable package delivered via opkg.
I appreciate your hints and interest.
Just to be more specific, this my problem is more like feature request either ad-hoc for me, or for whole community. I don't feel comfortable to go deeper into development area of OpenWrt. I am user of the product without any kernel or system component development capabilities.
should be easy enough as they are already upstreamed in Linux kernel
so if you want packages it's just a matter of adding a makefile that defines a package with them, and I can do that. (adding an out-of-tree driver is way harder and I can't do that)
But I see the error "waiting for root device". That means the kernel cannot find the root filesystem so making a package is useless (as packages are installed in root filesystem, which is not accessible until the driver is loaded).
I think both drivers must be compiled-in aka enabled in kernel config of 64bit, which is even easier as I think I only need to add
Of course I cannot test this, if I compile images with it, and give you a download link, will you test them?
Do you want Luci web interface in the test images?
Note that even if all this works and I open a PR with this contribution it will take months before someone merges it and can be used without setting up your own Linux VM to run build locally. It's not hard to just build OpenWrt, it's very automated.
Bingo! Your image works without any flaw. I can confirm that it works on AWS t3 instances very well.
While comparing it with standard openwrt image, I see that you also implemented new kernel 5.4.87, while standard uses 4.14.209. That's great, but many opkg packages can't be installed.
But of course, this exercise is mainly for proof of concept, which was successful.
If you would need, for any kind of confirmation/verification/curiosity, I can give you details to ssh and luci web connection.
Many thanks for your help!
I did not implement that, I used OpenWrt master branch, aka the latest source code. It's roughly equivalent to "OpenWrt development snapshot". or a "nightly build".
That error about kernel versions when installing packages is normal for it. The packages in the repos are built by the official build bots every few days and may be too old or too new for what I compiled.
I used that because I will now open a PR to contribute this change in the master branch, so I wanted to make sure it works for that.
Since you will probably want to use it as a normal person I'll do the same thing with OpenWrt stable release 19.xxx source.
This way the kernel is the same version as normal stable release and it should let you install everything.
I'll be back in a bit with a new download link
yeah, just warn people using that "Community AMI" to NOT upgrade their OpenWrt system as that would "upgrade" the kernel too and remove the ability to use t3 instances. You can update packages all you want, but no kernel updates.
If all goes well and OpenWrt members accept my contribution (will open a PR in a bit), this should get merged in time for the next official stable release (which at this time is unknown, fun fact: they skipped 20.xxx release already)
But it will NOT go in official OpenWrt 19.07 stable releases.
Hmm, something is not OK, I am getting again the error "waiting for root....". The difference now is, that first image was ext4, now it is squashfs. And also difference is kernel version.
Could you please generate ext4 image?