Detecting user installed pkgs

I think this may be a reasonable solution for you:

Please see the new EDIT 2023-10-16 sections for an addition that my make this work fully.

Here is a solution for when you need to determine the list of installed packages beyond the default packages and not including packages that are just dependencies and you used the firmware builder etc. Rather than just provide a manual recipe to use the script I referenced a couple days ago, I modified it to minimize manual work.

You need to provide the list of default packages. That can easily be obtained from the list provided in the firmware selector web site in the section called "Customize installed packages and/or first boot script" or from the default_packages section of the profiles.json file for your target from the downloads.openwrt.org site.
Put the list, one name per line in file /tmp/my-def-pkgs .

Copy the following script to your openwrt device, possibly in /tmp/ or /root/ , set it as executable (chmod + x) and run it.

EDIT 2023-10-16(1) - added a proposed file name in the script text.

#!/bin/sh

# proposed script file name: listUserInstalled.sh
# added 2023-10-16

if test ! -s /tmp/my-def-pkgs 
 then 
   echo " /tmp/my-def-pkgs was not found or is empty."
   echo " Please add list of default packages there and re-run this script."
   echo
   exit
 fi

opkg list-installed >/tmp/my-wd-input

echo "made temp file for what-depends list at /tmp/my-wd-input"
echo "generating list of packages and what depends on each to file /tmp/my-wd-output..."

for i in $( cat /tmp/my-wd-input | cut -d\  -f 1 | sort -u )
  do 
    #echo $i
    printf "."
    k=$( opkg whatdepends $i | grep  "^\t" | sed 's/^\t//g' | cut -d\  -f1 | sed 's/\n/ /g' )
    j=$( echo "$k" | wc -w )
    printf "$j\t$i\t $( echo $k )\n" >>/tmp/my-wd-output
    # printf "$j\t$i\t $( echo $k )\n"
    printf "."
done
echo ""

echo "preparing input list for user installed list..."
cat /tmp/my-wd-output | grep "^0" | cut -f2 >/tmp/my-user-inst-input

echo "The list of user installed packages is being written to /tmp/my-user-installed-packages"
echo "as well as on screen..."

for i in $( cat /tmp/my-user-inst-input)
  do
    if !(grep -Fqw $i /tmp/my-def-pkgs)
     then 
       echo $i >>/tmp/my-user-installed-packages
       echo $i
    fi
  done

echo
echo Done
echo "Clean up temp files /tmp/my-wd-output and /tmp/my-user-installed-packages in /tmp/ before re-running script."
echo

The opkg whatdepends function is slow so this process may take several minutes to run on slow processor but the script provides a progress indicator.

Once you have the list of user installed packages on your existing installation, you will probably have to do some manual research to determine what can be added in the new version as-is and what you might need to make accommodations for changes for.

EDIT 2023-10-16(2) - @R00 - Per your reply about finding 4 packages unaccounted for with the output of the script I provided above, I'm fairly sure I found the answer. :slight_smile:
The profiles.json file in the web directory for each target not only has the list of default packages but it also lists device specific packages in the profiles section. The few I checked which includes tagets different than your C7, like mips24 and aarch64 all matched.

I wrote a script that can either just provides the URL or can fetch it and then parse it to generate the "full" list of standard packages in /tmp/my-def-pkgs all ready to run the script from May 31.
I added a note in the above script as a suggested script name of listUserInstalled.sh .

I propose a name of prepToListUserInstalled.sh for this new script.
prepToListUserInstalled.sh gathers the board_name version, and target from ubus call system board and generates what should be a valid URL for the profiles.json file. It works on the 3 devices I tested with.

If you run prepToListUserInstalled.sh by itself, it prints some variable info and the URL so you could manually wget it.
If you add -full to the script call, it will wget the profiles.json file using the generated URL, extract the device and default packages and write the /tmp/my-def-pkgs file.
./prepToListUserInstalled.sh -full

If /tmp/my-def-pkgs already exists, the script quits without fetching or writing any files.
If /tmp/profiles.json aleady exists, the script skips the fetch and uses the existing file.
Manually delete or rename either files as desired to re-run the script and control input/output.

Here is proposed script prepToListUserInstalled.sh


# proposed script name: prepToListUserInstalled.sh

# also install script listUserInstalled.sh and chmod +x if needed
# and call it after running this script and checking output.

myFullRun="false"

if [ "$1" = "-full" ]
  then
    printf "\n Asked for full run!\n\n"
    myFullRun="true"
  else
    printf "\n Just running prep.\n\n"
fi

myDeviceName=$(ubus call system board | jsonfilter -e '@.board_name' | tr ',' '_')
printf "myDeviceName is "%s"\n" $myDeviceName

myDeviceTarget=$(ubus call system board | jsonfilter -e '@.release.target')
printf "myDeviceTarget is %s\n" $myDeviceTarget

myDeviceVersion=$(ubus call system board | jsonfilter -e '@.release.version')
printf "myDeviceVersion is %s\n" $myDeviceVersion

myDeviceJFilterString=@[\"profiles\"][\"$myDeviceName\"][\"device_packages\"]
printf "myDeviceJFilterString is %s\n" $myDeviceJFilterString

myDefaultJFilterString=@[\"default_packages\"]
printf "myDefaultJFilterString is %s\n" $myDefaultJFilterString

#myDeviceProfilesURL="https://downloads.openwrt.org/releases/$myDeviceVersion/targets/$myDeviceTarget/profiles.json"
if [ "$myDeviceVersion" = 'SNAPSHOT' ] ; then
    myDeviceProfilesURL="https://downloads.openwrt.org/snapshots/targets/$myDeviceTarget/profiles.json"
else
    myDeviceProfilesURL="https://downloads.openwrt.org/releases/$myDeviceVersion/targets/$myDeviceTarget/profiles.json"
fi

# 2023-10-17: Potential better way to get URL:
#myDeviceProfilesURL=$(grep openwrt_core /etc/opkg/distfeeds.conf / | grep -o "https.*[/]")profiles.json

printf "\nmyDeviceProfilesURL is %s\n\n" $myDeviceProfilesURL

if ! [ "$myFullRun" = "true" ]                                                                   
  then                                                                                           
    printf " You can manually wget the profiles.json file at the URL above\n"
    printf " to /tmp/profiles.json\n"
    printf " or\n"
    printf " Add -full to the command line to fetch the json file and output /tmp/my-def-pkgs.\n\n"

    exit 0                                                                                        
fi                                                                                               
                                                                                                 
                                                                                                 
if test -s /tmp/my-def-pkgs                                                                      
  then                                                                                           
    printf "\n File /tmp/my-def-pkgs already exists!\n"                                          
    printf " Since this script creates it, please remove it and run this script again.\n\n"      

    exit 0                                                                                 
fi                                                                                         
    
if test ! -s /tmp/profiles.json 
  then 
    printf "\n /tmp/profiles.json file was not found.  Fetching it.\n\n"
    wget -O /tmp/profiles.json "$myDeviceProfilesURL"
  else
    printf "\n /tmp/profiles.json was found.  Skipping the fetch!\n\n"
fi

if test ! -s /tmp/profiles.json
  then
    printf "\n\n OOPS! /tmp/profiles.json did not get created.\n\n"
    printf " Exiting!\n\n"
    exit 0
  else
    jsonfilter -i /tmp/profiles.json -e $myDeviceJFilterString | sed s/\"/''/g | tr '[' ' ' | tr ']' ' ' | sed s/\ /''/g | tr ',' '\n' > /tmp/my-def-pkgs

    jsonfilter -i /tmp/profiles.json -e $myDefaultJFilterString | sed s/\"/''/g | tr '[' ' ' | tr ']' ' ' | sed s/\ /''/g | tr ',' '\n' >> /tmp/my-def-pkgs

    printf "\n%s\n" " The list of packages, including device specific and default is:"
    cat /tmp/my-def-pkgs

    printf "\n%s\n" " The list was also written to /tmp/my-def-pkgs "
    printf "%s\n\n" " so the script to list user installed packages can be run now."
    printf "%s\n\n" " Manual cleanup of temp files in /tmp/ may be needed if you run this script again."
    printf "%s\n\n" " Manually run the script to list user installed packages next."
    printf "%s\n\n" " End of prep script."
fi
# end of script

Maybe this is a valid solution now.
End EDIT 2023-10-16(2)

I hope this is useful to you.

Please mark as solution if it provides what you need or ask for clarification on anything that isn't clear.

4 Likes