How to Make an Interactive TCP Server with NodeMCU on the ESP8266

Introduction

 

This is Part 2 of an advanced series on connecting to and working with the "Internet of Things." Part 1 is located here. This project is relatively straightforward but still involves working intimately with the Linux terminal. Before attempting this project, it's recommended to have some experience with Linux as well as the Lua scripting language. Here is some documentation on Lua  as well as the documentation for the NodeMCU API. Remember, docs are your friend!
Last time, we discussed how to customize and compile the NodeMCU firmware from source using open source tools. Assuming a working NodeMCU installation, we're now going to explore some of the actual functionality of the device and begin interacting with both the "internet" and "things" aspects of IoT. Starting with a fresh flashing of the NodeMCU firmware, this project will walk through connecting to the device with Screen and using the built-in Lua interpreter to set up the file system and connect to the network. We'll then look at automating the scripting process with Luatool and put it all together with a basic TCP server that can interact with a pulse-width modulated LED, a potentiometer, and a switch.

Supplies Needed:

• Linux Computer (Ubuntu 15.04 Vivid Vervet)
• NodeMCU Devkit v0.9 with the following user modules compiled: Node, File, GPIO, WiFi, Net, PWM, Timer, ADC, UART
• WiFi access 
• 1x LED
• 1x 100-300 Ohm resistor
• 1x 5K resistor
• 1x 10K potentiometer
• 1x Normally open momentary switch

Software:

• Git​
• Python 2.7
• Pyserial
• Linux Screen
• Luatool

​To install these in Ubuntu 15.04, type the following into the terminal:

sudo apt-get install git screen python2.7 python-serial

mkdir -p ~/.opt && cd ~/.opt # just a user-owned directory for software. Replace with what suits your needs

git clone https://github.com/4refr0nt/luatool.git

Connecting to NodeMCU for the First Time

1) Connect to NodeMCU:
All interaction with NodeMCU happens through the serial port at 9600 baud. I prefer to use Screen in the terminal but Minicom or any program that can interact with the serial port will do. Plug in your NodeMCU Devkit and type:

screen /dev/ttyUSB0 9600

If your NodeMCU Devkit happens to live on a different serial port than /dev/ttyUSB0, use dmesg | grep tty to find it.
This should show just a plain black terminal screen with no text. Once connected, press the button labeled USER to reset the device. Or type the following into the terminal and press enter:

node.restart()

Press this lil' guy right here

You should now get a bit of gibberish (communication at a different baudrate) and then some kind of welcome message and a '>' prompt. You are now inside NodeMCU's Lua interpreter. The init.lua error is expected since this is a brand new install and we haven't yet sent any scripts to the device. Init.lua is what NodeMCU runs once at startup (sort of like the setup() function in Arduino).

The interpreter's inside the computer?! It's so simple!

2) Format node file system
If you are working with a fresh install of NodeMCU, we need to format NodeMCU's file system before we can start writing and saving Lua scripts. In the interpreter, type:

file.format()

Once completed, you can see information on the file system by using file.fsinfo() call:

remaining, used, total = file.fsinfo()

print("\nFile system info:\nTotal: "..total.." Bytes\nUsed: "..used.." Bytes\nRemaining: "..remaining.." Bytes\n")

This will show you total size, free space, and occupied space of just the NodeMCU file system, not the raw memory information of the ESP8266. The ESP8266 has 64 KiB of instruction RAM, 96 KiB of data RAM and about 4 MiB of external flash memory.

Three and a half megabytes? We can work with that.

Note: If you see something like "stdin:1: unexpected symbol near ‘char(27)’” or “stdin: bad header in precompiled chunk” while you're typing commands, don't worry: it's just a little tic of Screen. Hannes Lehmann wrote about the issue on his website saying, "... If you get some errors like “stdin:1: unexpected symbol near ‘char(27)’” or “stdin: bad header in precompiled chunk” then your terminal doesn’t support backspace or arrow input (either copy&paste issue, or you have done a correction of your input). Don’t worry, just repeat the command."
My particular setup with Screen seems like it can handle corrections using backspace but it tends to mess up when I try use the arrow key.

Hello World, Hello WiFi!

3) Connect to WiFi network
Since the main selling point of the ESP8266 is its WiFi stack, let's use the Lua interpreter to connect to the local network and get an IP address.
The interactive Lua terminal on the device is good for prototyping small bits of code. To connect to your local WiFi and display the IP information, type into the terminal:

wifi.setmode(wifi.STATION)

wifi.sta.config("wifi_name","wifi_pass") -- Replace these two args with your own network

ip, nm, gw=wifi.sta.getip()

print("\nIP Info:\nIP Address: "..ip.." \nNetmask: "..nm.." \nGateway Addr: "..gw.."\n")

We're connected!

4) Automate with Luatool
Testing small pieces of code with the interpreter is great but what if you want to write something more complicated and have it run automatically at startup? Github user 4refr0nt wrote a program called Luatool that can upload Lua scripts from your computer to the NodeMCU Devkit and save them on the device's file system. Navigate to the Luatool folder that you cloned from Github in the beginning:

cd ~/.opt/luatool/luatool

It should have two files in it in addition to luatool.py: init.lua and main.lua. Using your favorite editor, modify the respective files to look like this:

                    -- init.lua --


-- Global Variables (Modify for your network)
ssid = "my_ssid"
pass = "my_pass"

-- Configure Wireless Internet
print('\nAll About Circuits init.lua\n')
wifi.setmode(wifi.STATION)
print('set mode=STATION (mode='..wifi.getmode()..')\n')
print('MAC Address: ',wifi.sta.getmac())
print('Chip ID: ',node.chipid())
print('Heap Size: ',node.heap(),'\n')
-- wifi config start
wifi.sta.config(ssid,pass)
-- wifi config end

-- Run the main file
dofile("main.lua")
                  

 

                    -- main.lua --


-- Connect 
print('\nAll About Circuits main.lua\n')
tmr.alarm(0, 1000, 1, function()
   if wifi.sta.getip() == nil then
      print("Connecting to AP...\n")
   else
      ip, nm, gw=wifi.sta.getip()
      print("IP Info: \nIP Address: ",ip)
      print("Netmask: ",nm)
      print("Gateway Addr: ",gw,'\n')
      tmr.stop(0)
   end
end)

 -- Start a simple http server
srv=net.createServer(net.TCP)
srv:listen(80,function(conn)
  conn:on("receive",function(conn,payload)
    print(payload)
    conn:send("

 Hello, NodeMCU!!! 
")
  end)
  conn:on("sent",function(conn) conn:close() end)
end)
                  

 

5) Close your current Screen session (Luatool can't communicate with NodeMCU otherwise) and then upload both files to the NodeMCU:

python luatool.py --port /dev/ttyUSB0 --src init.lua --dest init.lua --verbose

python luatool.py --port /dev/ttyUSB0 --src main.lua --dest main.lua --verbose

6) Reconnect to the NodeMCU Devkit with Screen and press the USER button to reset the device:

screen /dev/ttyUSB0 9600

You should then see something like this:

Go to that IP address (in my case 192.168.168.10), and Voila!

Check it out! We have a teeny, tiny server!

Now for Some Hardware

7) Build the circuit and upload the server code

Here is the schematic. It's fairly simple since we're dealing mostly with software.


The hardware laid out in Fritzing

My own setup

Now edit the init.lua and main.lua files from before to look like the following:

                    -- init.lua --

-- Network Variables
ssid = "your_ssid"
pass = "your_pass"

-- Byline
print('\nAllAboutCircuits.com NodeMCU Example\n')

-- Configure Wireless Internet
wifi.setmode(wifi.STATION)
print('set mode=STATION (mode='..wifi.getmode()..')\n')
print('MAC Address: ',wifi.sta.getmac())
print('Chip ID: ',node.chipid())
print('Heap Size: ',node.heap(),'\n')

-- Configure WiFi
wifi.sta.config(ssid,pass)

dofile("main.lua")
                  

 

                    -- main.lua --

----------------------------------
-- WiFi Connection Verification --
----------------------------------
tmr.alarm(0, 1000, 1, function()
   if wifi.sta.getip() == nil then
      print("Connecting to AP...\n")
   else
      ip, nm, gw=wifi.sta.getip()
      print("IP Info: \nIP Address: ",ip)
      print("Netmask: ",nm)
      print("Gateway Addr: ",gw,'\n')
      tmr.stop(0)
   end
end)


----------------------
-- Global Variables --
----------------------
led_pin = 1
sw_pin = 2
adc_id = 0 -- Not really necessary since there's only 1 ADC...
adc_value = 512

-- Amy from Gargantia on the Verdurous Planet
blink_open = "http://i.imgur.com/kzt3tO8.png"
blink_close = "http://i.imgur.com/KS1dPa7.png"
site_image = blink_open

----------------
-- GPIO Setup --
----------------
print("Setting Up GPIO...")
print("LED")
-- Inable PWM output
pwm.setup(led_pin, 2, 512) -- 2Hz, 50% duty default
pwm.start(led_pin)

print("Switch")
-- Enable input
gpio.mode(sw_pin, gpio.INPUT)

----------------
-- Web Server --
----------------
print("Starting Web Server...")
-- Create a server object with 30 second timeout
srv = net.createServer(net.TCP, 30)

-- server listen on 80, 
-- if data received, print data to console,
-- then serve up a sweet little website
srv:listen(80,function(conn)
 conn:on("receive", function(conn, payload)
  --print(payload) -- Print data from browser to serial terminal
 
  function esp_update()
            mcu_do=string.sub(payload,postparse[2]+1,#payload)
            
            if mcu_do == "Update+LED" then 
             if gpio.read(sw_pin) == 1 then
              site_image = blink_open
              -- Adjust freq
              pwm.setclock(led_pin, adc_value)
              print("Set PWM Clock") 
          elseif gpio.read(sw_pin) == 0 then
           site_image = blink_close
           -- Adjust duty cycle
           pwm.setduty(led_pin, adc_value)
           print("Set PWM Duty") 
             end  
            end
            
            if mcu_do == "Read+ADC" then
             adc_value = adc.read(adc_id)
             -- Sanitize ADC reading for PWM
    if adc_value > 1023 then
     adc_value = 1023
    elseif adc_value < 0 then
     adc_value = 0
    end
    print("ADC: ", adc_value)
            end
        end

        --parse position POST value from header
        postparse={string.find(payload,"mcu_do=")}
        --If POST value exist, set LED power
        if postparse[2]~=nil then esp_update()end


  -- CREATE WEBSITE --
        
        -- HTML Header Stuff
        conn:send('HTTP/1.1 200 OK\n\n')
        conn:send('\n')
        conn:send('\n')
        conn:send('\n')
        conn:send('

\n') conn:send('

ESP8266 Blinker Thing!

\n') -- Images... just because conn:send(''" WIDTH="392" HEIGHT="196" BORDER="1">

\n') -- Labels conn:send('ADC Value: '..adc_value..'
') conn:send('PWM Frequency (Input High): '..adc_value..'Hz') conn:send('or
') conn:send('PWM Duty Cycle (Input Low): '..(adc_value * 100 / 1024)..'%
') -- Buttons conn:send('

\n'

) conn:send('\n') conn:send('\n') conn:send('\n') conn:on("sent", function(conn) conn:close() end) end) end)

 

Again, close any active Screen sessions for Luatool and upload both files to the NodeMCU:

python luatool.py --port /dev/ttyUSB0 --src init.lua --dest init.lua --verbose

python luatool.py --port /dev/ttyUSB0 --src main.lua --dest main.lua --verbose

8) Reconnect to the NodeMCU Devkit with Screen and press the USER button to reset the device:

screen /dev/ttyUSB0 9600

What's Going On Here?

When the user presses the "Read ADC" button in the browser, the browser is updated with the current ADC reading of the potentiometer and that value is posted to the NodeMCU's serial terminal if you have it open. If the pushbutton is not pressed, the input pin is pulled high which means that the current ADC reading will be used to set the LED's PWM frequency. If it is pressed, and the input pin is pulled low, the LED's PWM duty cycle is adjusted. You also get a different image in the browser depending on what LED parameter is being set.
Now let's take a moment to dig through the code to see how this is implemented. Init.lua contains mostly code from the "Hello Word, Hello WiFi" section. It displays some information about the chip and connects to the wireless network. Main.lua is where all the fun happens -- it's a modification of the code here. The flow of that script is to print IP information, initialize some global variables, configure the I/O, and then create a TCP server that listens on port 80. Any time a button is pressed, an HTTP POST method is called by the browser. The string.find() method in the script looks though the HTTP header and tries to find any mention of a button named "mcu_do". If this does turn up, the esp_update() function is called and depending on the value assigned to mcu_do, it will either read the ADC or update the parameters of the LED. And there you have it, a bit of hardware that can interact with a browser in a meaningful way and vice versa.

Closing Remarks

This project only scratches the surface of what you can do with the ESP8266 and NodeMCU. It can act as an MQTT broker, talk UDP as well as TCP, perform cryptography, communicate with peripherals over I2C and SPI, and a ton more. The NodeMCU Devkit is a powerful hardware device that can enable very powerful IoT applications but is by no means the only or even the best solution for all projects. Keep your eyes peeled as new solutions in software and hardware spring into the blossoming IoT landscape.