OpenWrt $(TMP_DIR) directed to disk? $(TOPDIR)/tmp

For Developers
1

I went looking through base code and noticed that $(TMP_DIR) and $(TMPDIR) are both pointed to $(TOPDIR)/tmp. This writes temporary files to disk, since $(TOP_DIR) is the OpenWrt buildroot.

Is there a reason this isn't pointing to RAM? Is there a reason I shouldn't remove ./tmp as a directory and symlink it to /tmp?

2

Tmp contains also reusable package and dependency data. See the files starting with dots.

(Those are automatically recreated, if missing, but their presence can still speed up the compilation process a bit).

1 Like
3

Right, I saw the target and kernel temp files.. but, Would it be faster to use RAM for temp file stuff than even an SSD can offer? Even if it has to re-create the .kconfigxxxx and package index? Does OpenWrt not use $(TMP_DIR) for processing at all?

4

yes... it's mostly the io from build_dir/targ*/lin*/tmp @ image.mk where bin_dir is on the same disk...

( edit: technically if you enable ccache and also place that on ramdisk... you'll see quite a speed increase in subsequent builds, the rest of the stuff as mentioned is often used on subsequent builds to actually speed things up by not performing stuff that hasn't changed... )

1 Like
5

I was facing a situation where I wanted to use $(TMP_DIR) for compat rather than trying to call /tmp for processing archival files but then realized all of the unnecessary IO calls made me stop and question it.

6

imagebuilder with 300+ packages... @ 1GB rootfs ramdisk saves a grand total of 20 seconds... all during image.mk

################################## RAMDISK
1006632960 bytes (1.0 GB, 960 MiB) copied, 1.96355 s, 513 MB/s
Calculating checksums...
real    2m26.798s


################################## sata3 wdred
1006632960 bytes (1.0 GB, 960 MiB) copied, 9.87557 s, 102 MB/s
Calculating checksums...
real	2m44.742s
1 Like
7

Ah, I don't use imagebuilder or the SDK. I'll have to do some testing then. :slight_smile:

1 Like
8

test with ccache... then test with ccache on ramdisk...

you can get some good stats with '-s';

sample-ccache-stats 
# ccache -s

cache hit (direct)                 47845
cache hit (preprocessed)            7209
cache miss                         21268
cache hit rate                     72.13 %
called for link                      603
called for preprocessing            2890
compiler produced empty output      2241
preprocessor error                  7850
unsupported code directive           559
no input file                      10356
cleanups performed                     0
files in cache                     59147
cache size                           2.0 GB
max cache size                      15.0 GB
1 Like
9

I actually don't use ccache either :flushed: I ran into problems with ccache when I was building all the packages, and it's just been easier to forgo it.. I'll have to try it again. I do enough scratch compiles I can test it a variety of ways, I suppose. :smile:

1 Like
10

yeah... I also had some issues with ccache when something major changes in the source...

guess there is a point where there is a need for a ccache flush...?

1 Like
11

@jow said ccache was disabled on the buildbots because it was more trouble than it was worth, but my issue was a limit issue with luci and CONFIG_ALL=y. I just never turned it back on once I stopped having to build everything out like that. I'll give it another shot.

1 Like