OpenWrtAI with internet radio, local media, duckgo search and chatGPT

for those who may wish to build a OpenwrtAI. here are a few scripts to get you started . you will need to install python speech recongnistion and TTS
see this thread for details on how to do that.
these are the basic scripts that are require.
speak script:

!/usr/bin/env python
# need gTTS and mpg123
# pip install gTTS
# opkg install mpg123

from gtts import gTTS
import os
import sys

# define variables
s = sys.argv[1]
file = "/tmp/audio.mp3"

# initialize tts, create mp3 and play
tts = gTTS(s, 'com')
tts.save(file)
os.system("mpg123 " + file)

read script

#!/bin/bash

while read -r line;do
    echo -e "$line"
    ./google $line
done <$1

google - part of read -

#!/bin/bash
python3 speak "$*" #> /dev/null 2>&1

gtrans

#!/usr/bin/env python

import speech_recognition as sr

r = sr.Recognizer()
with sr.AudioFile('/tmp/mic.wav') as source:
    audio = r.record(source, duration=60)

command = r.recognize_google(audio)

text_file = open("Output.txt", "w")
text_file.write(command)
text_file.close()

AI - this is the bash script that you run. to be enabled by a button fuction, i use a zigbee button and zigbee keychain button( via domoticz) to run it, simply push it. then OpenWrtAI will initiate - you have pure privacy . unlike other AI's they listen to everything you say all the time and are log on remoter severs. OpenWrtAI will only provide information when you want it too.

AI-

#!/bin/bash

chroot /root/VLC_JULIUS/ /bin/bash ./kill.sh
pkill mpg123

A=0
B=0



while [[ $A < 3 ]]; do

wget -q --spider http://google.com

#______________onlineor offline speech engine________________#

if [ $? -eq 0 ]; then
      if  [ $B -eq 0 ];then
          python speak " what can I do for you" > /dev/null 2>&1
          B=1
       fi
arecord -f S16_LE -d 3 -r 8000  /tmp/mic.wav  > /dev/null 2>&1


   #espeak "one moment" > /dev/null 2>&1
   python speak "one moment"  > /dev/null 2>&1

   ./gtrans.py > /tmp/stt # > /dev/null 2>&1

   Output=$(cat /tmp/stt | sed -s '1d' | grep transcript | head -n 1 | sed  "s/'transcript': //g" |sed  "s/},//g" |sed 's/^ *//' |sed 's/}],//' | sed 's/{.*, //' ) # 
   #   echo "Online"


else
      if [ $B -eq 0];then
         espeak " what can I do for you" > /dev/null 2>&1
         B=1
       fi

    arecord -f S16_LE -d 3 -r 8000  /tmp/mic.wav  > /dev/null 2>&1


    espeak "one moment" > /dev/null 2>&1
     rm /tmp/stt2
    python test.py /tmp/mic.wav >>/tmp/stt2 
    cat /tmp/stt2 | grep '"text" :' | sed 's/ "text" : //g' | sed 's/""//g' | sed 's/"the"//g' | sed  '/^\s*$/d'

    voice=espeak
    #  echo "Offline"
fi

#__________________END _________________________#

echo $Output
NO_WHITESPACE="$(echo -e "${Output}" | sed 's/$/-1/' | tr -d ' ' | sed "s/'//g" )"
echo $NO_WHITESPACE

####################### PHRASE FUNCTION ###########

if [ $NO_WHITESPACE == 'playmusic-1' ]
      then
   
         echo "playing "
         cd /MEDIA/Audio
         find  -iname '*.mp3' | mpg123 -Z -@ -
         A=3

elif [ $NO_WHITESPACE == 'playCityFM-1' ]
      then

         echo "playing  citi fm "
         chroot /root/VLC_JULIUS /bin/bash VLC.sh 'http://rogers-hls.leanstream.co/rogers/win921.stream/icy' 
         sleep 2
         mpg123 http://127.0.0.1:9090/stream

         A=3

elif [ $NO_WHITESPACE == 'playKissFM-1' ]
      then

         echo "playing  kiss fm "
         chroot /root/VLC_JULIUS /bin/bash VLC.sh 'https://rogers-hls.leanstream.co/rogers/win1023.stream/playlist.m3u8'
         sleep 2
         mpg123 http://127.0.0.1:9090/stream
         
         A=3

elif [ $NO_WHITESPACE == 'playPeggyFM-1' ]
      then

         echo "playing  peggy fm "
         chroot /root/VLC_JULIUS /bin/bash VLC.sh 'http://live.leanstream.co/CJGVFM?'
         sleep 2
         mpg123 http://127.0.0.1:9090/stream

         A=3

elif [ $NO_WHITESPACE == 'playPower97-1' ]
      then

         echo "playing  power  fm "
         chroot /root/VLC_JULIUS /bin/bash VLC.sh 'https://live.leanstream.co/CJKRFM'
         sleep 2
         mpg123 http://127.0.0.1:9090/stream

         A=3

elif [ $NO_WHITESPACE == 'playVirginRadio-1' ]
      then

         echo "playing  power  fm "
         chroot /root/VLC_JULIUS /bin/bash VLC.sh 'https://18183.live.streamtheworld.com/CKMMFM.mp3'
         sleep 2
         mpg123 http://127.0.0.1:9090/stream

         A=3



elif [ "$NO_WHITESPACE" == 'weatheralert-1' ] || [ "$NO_WHITESPACE" == 'weatheralerts-1' ]
     then
         echo " weather alert"
         ./read /tmp/wstatement
          A=3

elif [ "$NO_WHITESPACE" == 'readbook-1' ] 
     then
         python speak " I have a selection of books that i can read -  which one would you like"
         python speak " dracula"
         python speak "emma"
         python speak "frankenstein"
     sleep 2
         python speak "which book would you like?"
     while [[ $A < 3 ]]; do
         arecord -f S16_LE -d 3 -r 8000  /tmp/mic.wav  > /dev/null 2>&1
         python speak "one moment"  > /dev/null 2>&1
         ./gtrans.py > /tmp/stt # > /dev/null 2>&1
         Output=$(cat /tmp/stt | sed -s '1d' | grep transcript | head -n 1 | sed  "s/'transcript': //g" |sed  "s/},//g" |sed 's/^ *//' |sed 's/}],//' | sed 's/{.*, //' ) #
         echo $Output
         N_WHITESPACE="$(echo -e "${Output}" | sed 's/$/-1/' | tr -d ' ' | sed "s/'//g" )"
         echo $N_WHITESPACE
     if [ "$N_WHITESPACE"   == 'Frankenstein-1' ]; then
          python speak " okay,  I will read frankenstien"
          sleep 2
           ./read ebook/frankenstein
     elif [ "$N_WHITESPACE" == 'Emma-1' ]; then
          python speak " okay,  I will read emma"
          sleep 2
           ./read ebook/emma
     elif [ "$N_WHITESPACE" == 'Dracula-1' ]; then
          python speak " okay,  I will read dracula"
          sleep 2
           ./read ebook/dracula
      else 
            echo "oops---1"
         if [ $A == 2 ]; then
          sleep 2
         
           ((A++))
        else
            python speak "sorry, Please repeat "
         ((A++))
          fi
      fi
   done
 
 else

       echo "oops"
    if [ $A == 2 ]; then
        sleep 2 
        python speak " Sorry I do not understand, good bye" 
         ((A++))
     else
        python speak "sorry, Please repeat "
        ((A++))
     fi


fi
############## END of PHRASE ###################
done 
######

now since openwrt mediaplayers are geting a little long in the tooth. they work with very few media streams to get around this we use VLC in chroot to transcode to an appropriate codex
for this you need to create the userspace

#!/bin/sh

THIS_CHROOT="/root/VLC_JULIUS"

#chroot is not there, build it
if [ ! -d "$THIS_CHROOT" ]; then
  debootstrap --arch amd64 bullseye "$THIS_CHROOT"

  chroot "$THIS_CHROOT" /bin/bash -c "apt install -y curl  build-essential git vim libsdl2-dev libasound2-dev zlib1g-dev wget tar unzip libsndfile1 libsndfile1-dev vlc"
fi


chroot "$THIS_CHROOT" /bin/bash -c "mount -t proc proc /proc && \
mount -t sysfs sys /sys && \
apt-get install -y vlc && \
mkdir /opt/julius && \
git clone https://github.com/palles77/julius.git && \
cd julius && \
chmod +x configure && \
./configure --enable-words-int --prefix=/opt/julius --enable-fork && \
make && \
make install && \
wget https://versaweb.dl.sourceforge.net/project/juliusmodels/ENVR-v5.4.Dnn.Bin.zip && \
unzip  ENVR-v5.4.Dnn.Bin.zip -d /opt/julius && \
rm ENVR-v5.4.Dnn.Bin.zip && \
wget http://www.repository.voxforge1.org/downloads/Nightly_Builds/AcousticModel-2021-01-25/HTK_AcousticModel-2021-01-25_8kHz_16bit_MFCC_O_D.zip && \
unzip HTK_AcousticModel-2021-01-25_8kHz_16bit_MFCC_O_D.zip -d /opt/julius/vox && \
rm HTK_AcousticModel-2021-01-25_8kHz_16bit_MFCC_O_D.zip"

you can stop the build after vlc install.. as the rest is related to julius for offline translation

apt-get install -y vlc && \  just delete everything after it and  remove && \ and changing it to " 

then you need to put these two scripts into your chroot folder
kill.sh

#!/bin/bash
pkill vlc  && exit

and VLC.sh

#!/bin/bash

sudo -H -u nobody cvlc $1  --sout="#transcode{vcodec=none,acodec=mp3,ab=64,channels=2,samplerate=44100}:http{mux=mp3,dst=:9090/stream}" --sout-keep --loop  >/dev/null 2>&1 & echo "" >  /tmp/tmp

exit 0

the hardest thing about getting internet radio to work is finding a site to scrape it from usually you find them in the page source page.
the easiest way when determining what you need as a call condition, is run AI script and when it ask you what you want to do, say it . and it will display what it thinks you are saying. it will ask you three times . just make sure the if condition matches also it is case sensitive

some of it related to my mycroft skills that one can find on github that I am adapting for this such as my gutenburg skill EVCAN skill if you live in Euope you use my metro weather skill for weather alerts

but all in all the AI works well and easy to set up . Mycroft can really annoying beeping allbtimes day and night . just to figure out how to get julius to be consistent for offline usage

okay have fun !

1 Like

okay as an update for offline speech to text. while julius you can configure to the very smallest foot print ( much much less then 50 meg of memory and only 4 meg of storage space and near instant translation ) but it requires you to configure to only recognize the words in your AI if conditions and the use of is grammar . so you have to configure phrase in a specific way.. since julius can not determine the first word accurately. you use grammar structure . ( but for translating large wav file in to text it out preform all of them on a normal cpu)
so it can guess the first word
ie phrase "play peggy FM" with the use of grammar

 play ------> city FM
         ------> power 97  since every thing is very dissimilar  it work well
        ------>  peggy fm
so it will guess the result as  "play  peggy"  quite reliably  but it will only guess what ever in your dictionary  so it will either guess "play peggy"  95% of the time or it might one of the other ones  some of the time but rarely 
it get trickier for "turn on" and "turn off"
turn off  ----> office light     
             ----> kitchen light 
turn on ----->office light
            -----> kitchen light     this will be  less reliable as they are too similar in sentence structure  it like a 50/50 chance of getting it right 

I tried alot of speech to text the most reliable is whisper but you need to run it through chroot as well

./whisper /opt/julius/bin/test.wav  --model tiny --language English   --fp16 False   -o /tmp -f txt

it will translate reasonably okay for 3 sec of recording it take 10 second on a normal cpu
if you want to use it a speech translator ie translate German to English

./whisper /opt/julius/bin/test.wav  --model small --language de --task translate  --fp16 False   -o /tmp -f txt

it do it but take a bit of time +30 seconds

well that make chroot more useful. as I was piping the audio and between chroot and host. but i was hunting for a better way and I found it ( sure not much info on chroot out there). for any one who interested.. but you can get direct access to all host devices simply by

mount -o bind /dev /chroot_mnt_point/dev

now I have direct access to the openwrt host devices and everything plays natively from within chroot .

which now means i can pretty much install anything a want now with little issue on openwrt , could even install mycroft AI. but honestly ,I am finding my AI simpler and easier and 3 times quicker plus I really hate python, is such a slow bloated way to do stuff only thing more bloated is node.js . A prime example of that is my gutenburg or recipe skill for mycroft. i wrote it in python using beautiful soup. it would take 10 - 20 minutes to process. what my shell scripts would do in under 5 minutes - to scrape 100s of pages of books and recipes and process them into usable formats. so it was back to shell scripts ( end of rant on python ha ha )

okay here the relatively finished version of openwrtAI still little bits here and there

it has around 100 public domain books that it will read to you , it has demonstration of how to install voice commands for domoticz, weather and emergency alerts that can be adjust for your area , book function uses fuzzy logic for book search so as long as you are close it should bring up the correct book, you can pause and restart the books where ever you left off and if power outage it remembers the last place you were, plus chapter jumping ( I still have to migrate the get book option from my mycroft skill) . probably one of the most significant difference with my AI and others is that I support multiple room play back via different 5.1 7.1 or greater audio device ports ( up to 32 channels) one openwrtAI router can do the entire house ai functions

todo add wolfarm and possible wiki info section

small update I forgot to update for whisper speech engine and or kaldi docker container. both function properly now

and updated version that has better audio channel control , I've tested up to 6 channels of simultaneous output
it has fuzzy logic on the main program as well simplifying conditions and making the device understand you much better with less re prompts.
. I currently mainly use it for internet radio and playing stored audio files and that I I capable to tell it were I want it to play to 3 separate locations ( you need a 7.1 audio device ) - it auto loads SST server kaldi if there a network outage and can run offline easily , then if the network comes online again it will shut down the the STT server automatically

small update to openwrtAI add in information. tab. say "information"- then ask "what information?" and you can ask it almost anything . example: "where is Mumbai" - "the population of Singapore" who is byd cars " or even "price flights Winnipeg to Mexico"

another small update - "information chat bot", when you enter information mode it also enter chat bot mode you can continuously ask questions. if you want more complex question you probably have to use kaldi server to transcribe as the python speech.recognition is limited to 5 -6 words. but works well enough for basic questions

added chatGPT chat box it works nice for simpler questions, but can give very long answers that are complete gibberish or completely unrelated topic.. but I guess chatGPT is still "learning"

another update to the AI added audiobook section. stored under /MEDIA/AudioBooks/Novels
and stored under /MEDIA/AudioBooks/Short_Stories/ sub group SSF & HORROR you can get the audiobooks from librivox

i also added in elevenLab speech-synthesis for very human like speech or you can clone a voice if you want the AI to speak like John Wayne or a lost loved one if so inclined the voices will be cached to the hardrive . the free elevenlab account allows 10000 character per month. caching it speeds up the voice creation but also you only need to create the sample once dramatically reducing elevenlabs web usage

based on AQARA zigbee cube as the switch

OpenWrtAI usage

    • flipping cube 90 degree – initiates the AI
    • flipping the cube 180 – stops the AI
    • rotating the cube right plays radio in the office
    • rotating the cube left plays radio in the house
      
     VOICE Commands 
        ◦ “Stop”  voice command to stop AI
        ◦ Information” for duck duck go search chat box
        ◦ “CHAT”    for ChatGTP  chat box
        ◦ “Sci-Fi”  Random play back   of Sci-fi audio book
        ◦ “Horror”  Random Play back of Horror audio book
        ◦ “continue”   reading a book where you left off
        ◦ “continue audiobook”  will continue play back of audio book
        ◦ “configure house” set default playback to entire house
        ◦ “configure living room”  playback  in living room
        ◦ “configure office” playback in office
        ◦ “weather update”  gets current weather conditions
        ◦ “weather forecast”  gets 7 day forecast
        ◦ “weather Alert”  tells what the weather alert  are 
        ◦ “Play Music”  local play back of stored music
        ◦ “Play Peggy FM”
        ◦ “Play City FM”
        ◦ “Play Virgin Radio”
        ◦ “Play kiss FM”
        ◦ “Play Power 97”
        ◦ “Play Mix FM”
        ◦ “Play CJOB”
        ◦ “Play TSN”
        ◦ “Play Top Hits”
        ◦ “Play Comedy”
        ◦ “Play UK Radio”
        ◦ “Play Frankfurt Radio”
        ◦ “Play Berlin Radio”
        ◦ “Read Book”  enters book section
                1. “list books” list book collection
                2.  “ chapter #”  jumps to book chapter
                3. “read ‘Title of book ‘ ”  will begin reading that book
        ◦ “Audio Book” enters audio section
                • “Chapter #”   jumps to specified chapter
                • “List Books”
                • “Random short stories” plays random short stories 
                • “Random Sci-fi” play randon scifi novel
                • “Random Horror” Plays random Horror Novel
                • “Random Novel” Plays Random Novel
                • “ Read ‘Title of Book’ ” 
                   
      Domoticz (Home Automation)
        ◦ “turn off/on office lights”
        ◦ “bathroom on/off” bathroom outlets
        ◦ “office on/off”   office outlets 
        ◦ . . . etc

here another update to openwrtAI with tft touch screen interface control

see here for more info : https://github.com/krywenko/ESP32-8048S070_smarthome_ai

1 Like

update to the above interface for openwrtAI .

video of interface inaction with openwrtAI you can use the screen or voice control both work equally as well

more info found here