WiFi testing program

TheWebMachine (with some help) has expanded this into a fully featured app for the Ringo: WiFiTest
To install it, just download the following zip file and unzip it to the root of your Ringo’s SD card:
https://github.com/TheWebMachine/TWM-Ringo/archive/master.zip
You can read the manual and see the full source on GitHub:

Discussion about the origin and evolution of this app can be found below; feel free to participate:


Over in the Support category a member was recently having trouble connecting to WiFi to update firmware: Can’t connect to wifi. It occurred to me that the firmware update is not the ideal way to debug WiFi problems. So I wrote a small WiFi application where the code can be easily fiddled with to try various potential solutions to WiFi problems.

Here is the program:

// Programming example: connecting to a WiFi network and periodically exchanging UDP datagrams with the
// NTP server at time.nist.gov (port 123). UDP is an unreliable protocol in that no UDP datagram is guaranteed
// to reach its destination, so not every request for current time will be honored. Furthermore time.nist.gov
// does not allow requests less than 4 seconds apart. So this program requests the time approximately every
// 10 seconds and extrapolates the time after each 10-second request by using the jiffy clock function millis().

// This program is intended to allow debugging WiFi connection problems including DNS failures. See the highlited
// code below where the user may customize this program to use either the DHCP-supplied DNS server or a known
// good DNS server, and to attempt to connect to a NIST NTP server either by name or by IP address. If the
// connection by name fails to resolve the name, then the DNS server in use is failing.

// If the WiFi connection to the specified SSID with the specified password succeeds, and the NTP server name
// resolves, the program will display the date/time returned by the server. If datagrams get lost, there may be
// momentary pauses in updating the date/time while the UDP request is retried; occasional datagram loss is
// expected as such is the nature of UDP.

// See highlighted comments below to customize program for your network and DNS testing.

// Example coded 5/10/2020 by Frank Prindle.

#include "MAKERphone.h"
MAKERphone mp;

void setup()
{
  mp.begin(1);

  /*-----------------------------------------------------------*/
  /* Set the following two strings to match your WiFi network. */
  /*-----------------------------------------------------------*/
  char *SSID = "FrankNet5";
  char *WPAPassword = "1234ABCD";

  /*-------------------------------------------------------------------*/
  /* Disable the following line to use DHCP supplied DNS server.       */
  /* Enable the following line to use a well-known DNS server.         */
  /*    Set the first IP address to a valid static IP on your network. */
  /*    Set the second IP address to the IP address of your router.    */
  /*    Set the third IP address to your network mask.                 */
  /*    Leave the fourth IP address alone (Google DNS).                */
  /*-------------------------------------------------------------------*/
  //WiFi.config(IPAddress(192,168,1,177),IPAddress(192,168,1,1),IPAddress(255,255,255,0),IPAddress(8,8,8,8));
  
  WiFi.begin(SSID,WPAPassword);
  int count=100;
  while(WiFi.status() != WL_CONNECTED && count--) delay(100);
  mp.display.setTextFont(1);
  mp.display.fillScreen(TFT_BLACK);
  if(WiFi.status() != WL_CONNECTED)
  {
    Serial.printf("WiFi cannot connect to given SSID with given password\n");
    statusline("WiFi Connect Failure", true);
    delay(5000);
    statusline("WiFi Connect Failure", false);
  }
  else
  {
    Serial.printf("WiFi is connected\n");
    statusline("WiFi Is Connected", true);
    delay(2000);
    statusline("WiFi Is Connected", false);
  }
}

void loop()
{
  unsigned int localPort = 8888; // Fairly arbitrary
  unsigned char inPacket[48];
  // NTP time request packet
  unsigned char outPacket[48] = {0b11100011, 0, 6, 0xEC, 0, 0 ,0, 0, 0, 0, 0, 0, 49, 0x4E, 49, 52, 0, 0, 0, 0, 0,
                                 0, 0, 0, 0 ,0 ,0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
                                 
  WiFiUDP udp;
  udp.begin(localPort);
  statusline("Trying To Resolve Server", true);

  /*-------------------------------------------------------------*/
  /* Enable only one of the following two lines.                 */
  /* Enable first line to use DNS to find an NTP server by name. */
  /* Enable second line to use an NTP server's IP address.       */
  /*-------------------------------------------------------------*/
  udp.beginPacket("time.nist.gov", 123); // NTP requests are to port 123
  //udp.beginPacket("129.6.15.28", 123); // NTP requests are to port 123

  statusline("Trying To Resolve Server", false);
  udp.write(outPacket, sizeof(outPacket));
  udp.endPacket();
  int count=100;
  while(udp.parsePacket() < sizeof(inPacket) && --count)
  {
    statusline("NTP Waiting For Response", true);
    delay(20);
  }
  statusline("NTP Waiting For Response", false);
  if(count)
  {
    // NTP request honored - time is in packet
    unsigned long ms = millis();
    udp.read(inPacket, sizeof(inPacket));
    unsigned long secsSince1900 = (inPacket[40]<<24) | (inPacket[41]<<16) | (inPacket[42]<<8) | inPacket[43];
    long secsSinceEpoch = secsSince1900 - 2208988800UL;

    // Extrapolate displayed time over 10 seconds
    while(millis()-ms < 10000)
    {
      // Extrapolate time now
      long sse = secsSinceEpoch+(millis()-ms)/1000;
      char *msg = ctime(&sse);
      msg[24]='\0';
      Serial.printf("%s UTC\n",msg);

      // Display the extrapolated time - if top line is yellow, time is from NTP - if green, time is exratpolated
      if(sse == secsSinceEpoch) mp.display.setTextColor(TFT_YELLOW);
      else                      mp.display.setTextColor(TFT_GREEN);
      mp.display.fillScreen(TFT_BLACK);
      mp.display.setCursor(20,8);
      mp.display.print("FROM: time.nist.gov");
      mp.display.setTextColor(TFT_GREEN);
      mp.display.setCursor(8,50);
      mp.display.print(msg);
      mp.display.print("\n\n            UTC");
      mp.display.pushSprite(0,0);
    }
  }
  else
  {
    // NTP request not honored (either outgoing packet or incoming packet lost)
    statusline("NTP No Response - Retry", true);
    delay(1000);
    statusline("NTP No Response - Retry", false);
  }
  // Shut down UDP in preparation for next loop
  udp.stop();
}

// Display (on==true) or erase (on==false) transient status line near bottom of display
void statusline(char *msg, bool on)
{
  mp.display.setTextColor(on ? TFT_YELLOW : TFT_BLACK);
  mp.display.setCursor(0,100);
  mp.display.print(msg);
  mp.display.pushSprite(0,0);

}

You can compile and upload this to the Ringo phone using the Arduino IDE after customizing it (see comments in code). If your WiFi is working well and you’ve used a valid WiFi network SSID and password, it will display the UTC date and time obtained from NTP server time.nist.gov. If the WiFi connection fails or time.nist.gov cannot be resolved to an IP address, a status line will so indicate, and you can modify the code to further explore the problem. You are obviously not limited to the code variations I’ve marked; feel free to change this in any way that benefits you and enhances your understanding of using WiFi from the Ringo phone.

This is a simple example using UDP over IP. There are other more complex protocols supported by the ESP32 WiFi library such as TCP and HTTP, which you may want to explore on your own.

2 Likes

I gave this a try on mine and it works great on its own.

I also tried to export it as a bin to add as an application on the SD card and it doesn’t show up in the list to launch. Not sure why. Would be great to call this on demand from the main OS.

1 Like

Hey there,

Here is the link on how to do that - https://www.circuitmess.com/world/guides/circuitblocks-11/making-a-.bin-file-299.

I guess that your folder name is not the same as the .bin file name, which is crucial in order for this to work. :slight_smile:

Keep in touch and hit me up if you don’t manage to solve it!

Robert

@robertCM oh jeez…of course, that is precisely the caveat I missed. As soon as I name the folder and .bin the same, all works fine. :man_facepalming: Thanks for catching that!

@frankprindle Do you have this on a git I can fork from? I’d love to collaborate on expanding this into a more expansive wifi utility. If not, I can throw it up on my git to allow others to build upon it.

I went ahead and added a new repo for my Ringo code:

I updated the original code above to allow the user to return to the main loader with B or Home, since I prefer to run this as an app on the SD for future development (the original code required reflashing to return to normal)

My next task is to prompt the user at the start to provide the SSID/WPA, instead of a hard code of the values. We can store this on the SD for future runs. Once that is done, it will be a better framework to build upon and for others to play with.

I’m still struggling a little with figuring out what all the mp.*() functions are. Is this documented in one comprehensive location? Been picking through source for what I’ve learned of them thus far…but we all know how tedious that can be. haha

After some generous cut/paste from settingsApp and such in the main codebase, I was able to add the prompts at the start to setup the wifi network, then it proceeds along to the original NTP test.

I tidied everything up a bit, created an icon, and uploaded my current “stable” .ino and .bin. So, you can just download the /WiFiTest repo, unzip it to the SD card, pop it in, and it will show at the bottom of the loader menu. :slight_smile:

This lays some basic foundation for more functionality to be added without things being hard coded. I’m going to work on a menu to specify DHCP or manual network config, NTP server, etc to show immediately after successful connection to a network. Then roll that up into it’s own menu-selected function (I’m probably going to “borrow” a LOT from the settingsApp code for this) so more tests can be added in addition to the NTP test.

TWM, I’ve been offline since last Wednesday due to a storm. Now that I’m back, I see you took this and ran with it - good for you - that’s what I was hoping would happen; I posted it as a little tease as to where one could go with a WiFi application.

@frankprindle Glad you survived the storm! I’ve made sure to give credit where due in the src and on the git. If you wanna jump in and help further, feel free to hop on my git and take a look at where I’ve gotten thus far. :grin:

Hey,

Good to see some active and solid work! Unfortunately, we don’t have our functions documented completely yet, but borrowing from the actual apps is the way to go!

If you don’t understand something feel free to hit me up here so I can explain how something works more thoroughly.

There is also a lot of material online regarding the work with ESP32 and WiFi connections so I’m sure you’ll be able to find a lot of different things in no time. :slight_smile:
Even Arduino ESP32 libraries have a lot of examples which can be used and which display how different protocols work. (here are some of them)

Cheers,
Robert

EDIT: Make that v0.5.0, which now saves your last used WiFi network and attempts to reconnect to it on app launch! It also solves the WiFi disconnecting on return from homePopup() issue.

I’ve just released v0.4.1 of my WiFiTest application. In addition to the wifi signal strength meter added in v0.3, there’s now a Telnet Chat Server applet (totally “stolen” from the official Arduino tutorials haha) for additional real time communications testing.

2 Likes

(I deleted my previous post because I managed to pick through enough github source to figure out what I needed.)

Welcome v1.0.0! Yeah, forget 0.6.0, et al…this latest edition makes for a rather complete app on its own and it deserves a proper release version. DHCP Settings menu is now live! I’ll just be adding more tests and squashing bugs moving forward…maybe make things a little prettier. I’m not digging the grey. :upside_down_face:

WiFiTest_DHCPSettings_screenshot

I’m thinking next up will be a Captive Web Portal (AP Mode with DNS and basic Web Servers) and maybe a more advanced Web Server that loads pages from the SD card (would operate as a normal client/server on the WiFi network). Of course, both of these will be based on existing Arduino examples out there. :man_shrugging:

1 Like

Great work TWM!

Really looks and works good, just tried it, everything is perfect! :smiley:

Maaaajor kudos to all the work you’ve done.

We’ll stay in touch!
Robert

1 Like

TWM, thanks for making this program. It has been very handy for testing my 3 DNS servers and 3 NTP servers (2 Raspberry Pis and one Windows machine).

One suggestion: when entering the NTP server name/IP, if the name is long, the data-entry box doesn’t scroll text left to accommodate additional characters at the right end, but rather the characters overwrite characters at the beginning of the line below (Erase). Needs a better mechanism for entering longer names (such as TIME.WINDOWS.COM in all capital letters). I think there’s an example of doing this in the stock WiFi setup code where the password is entered (originally it failed to allow longer passwords). I haven’t actually checked your program where the WiFi password is entered - it may or may not have the same issue.

Oh wow! Nice catch there, Frank! Just checked and WiFi password entry does, indeed, scroll long entries but NTP field does not in v1.1.1. I have fixed the NTP field in v1.1.2, which I just uploaded to GitHub. Because of the way I built the DHCP Settings page with multiple fields live updating, it wasn’t exactly straight forward how to get it in there - I had to do nearly double the work in code as was needed for WiFi password entry’s singular page field, but I did learn some more about mp.textInput() this evening that will definitely come in handy later (I’m thinking basic Telnet Client :smirk:).

So, onto my next problem, which I’ve bumped to the top of the to-do list, ahead of adding more tests - because I’m all about user convenience…

I’m currently beating my head against the wall trying to add an Update App option to the menu, so you can just select it and the latest .bin is saved to the SD card and the app reloads itself to the new version (via mp.updateFromFS())…easy peasy, right!? I’ve got most of the important logic sorted, however, I’ve run into a snag…

No matter what method I attempt to utilize, what tutorial I try to follow, what forum I peruse, I can’t get a competent routine for actually downloading the file (ANY binary file type, even a PNG image) from a HTTP server (ANY server…github, my own public file server which forces all file types to download stream - even .txt, even an apache running on my local network) and placing that file to the SD card. I’ve tried:

  • HTTPClient.GET() with:
    • With a buffer
    • With the writeToStream(&file) function
  • Alternative HttpClient in the same ways
  • Manually writing a HTTP GET request to a WiFiClient and trying to read the results
  • I even copied the commented out fetchUpdate() function from the Ringo source thinking maybe that was a good starting point for something at least partly tested on a Ringo device, even if commented out and not being used in current firmware

No matter how I attempt to perform a GET to download a file via HTTP (screw HTTPS and CAs for now), no matter which example I use, I can’t get any file data streaming down. I always end up with an empty file or an error. I send the GET and, in most cases, get a 200 OK response, but can’t seem to grab any data after this. It’s as if the WiFiClient just stops receiving.

Can someone supply a known working Ringo-specific example of downloading a binary file and saving it to the SD card? I think I’ve got the SD card stuff down, I just really need working HTTPClient code for Ringo HTTP GET. It seems to not work with any known examples I could find…and I’ve found/tried dozens so far. Is there perhaps some bug in the main Ringo source that is breaking the standard examples? @robertCM, any thoughts on this??


EDIT

I really think working update code would be handy for anyone making a Ringo [or even MakerBuino, etc] app or game. To be able to download app updates from a server on command would make the user’s life easier than having to pull the SD and copy over a new .bin every time I put out an update…which has been frequent lately. :rofl:

SIDE NOTE: I just noticed that you guys added my app to https://circuitmess.com/resources/creations !!! :joy: Thank you so much! I consider it an honor to be featured in such a way. That having been said, my Author name should read “TheWebMachine”, not “The Web Machine”. A minor typo, of course. Anywho, thank you for the spotlight on your site!

Oh, I see, you mean the calls to fetchUpdate() are commented out. Hmmmm…

EDIT: See next post!

Aha, fetchUpdate() is still called. If boolean 34 is set in EEPROM, it is called upon reboot by main.cpp. So you definitely need to base your code on fetchUpdate() because it does exactly the same thing except it is using https.

TWM, just tried this code (based on fetchUpdate()) and it worked perfectly (not much error checking here, just enough to muscle through if everything is as specified). For this to work, you need to create an empty directory named “wget” at the root of the SD card to receive the file. Once the program executes, it will have fetched into that directory the file “index.html” from one of the few web servers I could find on the internet that still serves unencrypted files. Obviously, change SSID and password to match your WiFi network.

Here you go:

#include "MAKERphone.h"
MAKERphone mp;

void setup()
{
  mp.begin(1);
  Serial.begin(115200);
  mp.display.setTextFont(1);
  mp.display.fillScreen(TFT_BLACK);
  Serial.printf("wget started\n");
  mp.update();
  
  char *SSID = "FrankNet5";
  char *WPAPassword = "12345678";
  WiFi.begin(SSID,WPAPassword);
  int count=100;
  while(WiFi.status() != WL_CONNECTED && count--) delay(100);
  if(WiFi.status() != WL_CONNECTED)
  {
    Serial.printf("WiFi cannot connect to given SSID with given password\n");
  }
  else
  {
    Serial.printf("WiFi is connected\n");
    HTTPClient client;
    SD.remove("/wget/index.html");
    File file = SD.open("/wget/index.html","w");
    client.begin("http://www.candlepowerforums.com/vb/index.html");
    int result = client.GET();
    if(result>0)
    {
      if(result == HTTP_CODE_OK)
      {
        client.writeToStream(&file);
        client.end();
        file.close();
        Serial.printf("File written\n");
      }
      else
      {
        Serial.printf("GET() failed with result %d\n",result);
      }
    }
    else
    {
      Serial.printf("GET() failed with result %d\n",result);
    }
  }
}

void loop()
{
  mp.loader(); //quit
}

TWM, I noticed that you said you were having trouble downloading a data file rather than an HTML file. I couldn’t quickly find a public data file that could be sent unencrypted, so I updated the program to https to download to the SD card a .ZIP file that was readily available. That also worked perfectly, though the download took longer than I might have expected given that the data file is less than 1.5MB, but I guess the Ringo doesn’t write to the SD card very quickly.

#include "MAKERphone.h"
MAKERphone mp;

void setup()
{
  const char* ca = \
  "-----BEGIN CERTIFICATE-----\n" \
  "MIIDxTCCAq2gAwIBAgIQAqxcJmoLQJuPC3nyrkYldzANBgkqhkiG9w0BAQUFADBs\n" \
  "MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\n" \
  "d3cuZGlnaWNlcnQuY29tMSswKQYDVQQDEyJEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\n" \
  "ZSBFViBSb290IENBMB4XDTA2MTExMDAwMDAwMFoXDTMxMTExMDAwMDAwMFowbDEL\n" \
  "MAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3\n" \
  "LmRpZ2ljZXJ0LmNvbTErMCkGA1UEAxMiRGlnaUNlcnQgSGlnaCBBc3N1cmFuY2Ug\n" \
  "RVYgUm9vdCBDQTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMbM5XPm\n" \
  "+9S75S0tMqbf5YE/yc0lSbZxKsPVlDRnogocsF9ppkCxxLeyj9CYpKlBWTrT3JTW\n" \
  "PNt0OKRKzE0lgvdKpVMSOO7zSW1xkX5jtqumX8OkhPhPYlG++MXs2ziS4wblCJEM\n" \
  "xChBVfvLWokVfnHoNb9Ncgk9vjo4UFt3MRuNs8ckRZqnrG0AFFoEt7oT61EKmEFB\n" \
  "Ik5lYYeBQVCmeVyJ3hlKV9Uu5l0cUyx+mM0aBhakaHPQNAQTXKFx01p8VdteZOE3\n" \
  "hzBWBOURtCmAEvF5OYiiAhF8J2a3iLd48soKqDirCmTCv2ZdlYTBoSUeh10aUAsg\n" \
  "EsxBu24LUTi4S8sCAwEAAaNjMGEwDgYDVR0PAQH/BAQDAgGGMA8GA1UdEwEB/wQF\n" \
  "MAMBAf8wHQYDVR0OBBYEFLE+w2kD+L9HAdSYJhoIAu9jZCvDMB8GA1UdIwQYMBaA\n" \
  "FLE+w2kD+L9HAdSYJhoIAu9jZCvDMA0GCSqGSIb3DQEBBQUAA4IBAQAcGgaX3Nec\n" \
  "nzyIZgYIVyHbIUf4KmeqvxgydkAQV8GK83rZEWWONfqe/EW1ntlMMUu4kehDLI6z\n" \
  "eM7b41N5cdblIZQB2lWHmiRk9opmzN6cN82oNLFpmyPInngiK3BD41VHMWEZ71jF\n" \
  "hS9OMPagMRYjyOfiZRYzy78aG6A9+MpeizGLYAiJLQwGXFK3xPkKmNEVX58Svnw2\n" \
  "Yzi9RKR/5CYrCsSXaQ3pjOLAEFe4yHYSkVXySGnYvCoCWw9E1CAx2/S6cCZdkGCe\n" \
  "vEsXCS+0yx5DaMkHJ8HSXPfqIbloEpw8nL+e/IBcm2PN7EeqJSdnoDfzAIJ9VNep\n" \
  "+OkuE6N36B9K\n" \
  "-----END CERTIFICATE-----\n";

  mp.begin(1);
  Serial.begin(115200);
  mp.display.setTextFont(1);
  mp.display.fillScreen(TFT_BLACK);
  Serial.printf("wget started\n");
  mp.update();
  
  char *SSID = "FrankNet5";
  char *WPAPassword = "12345678";
  WiFi.begin(SSID,WPAPassword);
  int count=100;
  while(WiFi.status() != WL_CONNECTED && count--) delay(100);
  if(WiFi.status() != WL_CONNECTED)
  {
    Serial.printf("WiFi cannot connect to given SSID with given password\n");
  }
  else
  {
    Serial.printf("WiFi is connected\n");
    HTTPClient client;
    //SD.remove("/wget/index.html");
 SD.remove("/wget/DnsServerPortable.zip");
    //File file = SD.open("/wget/index.html","w");
 File file = SD.open("/wget/DnsServerPortable.zip","w");
    //client.begin("http://www.candlepowerforums.com/vb/index.html");
 client.begin("https://download.technitium.com/dns/DnsServerPortable.zip",ca);
    int result = client.GET();
    if(result>0)
    {
      if(result == HTTP_CODE_OK)
      {
        client.writeToStream(&file);
        client.end();
        file.close();
        Serial.printf("File written\n");
      }
      else
      {
        Serial.printf("GET() failed with result %d\n",result);
      }
    }
    else
    {
      Serial.printf("GET() failed with result %\n",result);
    }
  }
}

void loop()
{
  mp.loader(); //quit
}
1 Like

But if you look, fetchUpdate() function itself is entirely commented out too. How can is it calling upon a function that isn’t even being compiled into the binary?

Ok, this is going to need an explanation. Where did this wget dir requirement come from? I can read/write to /WiFiTest/settings.txt just fine; why can’t I download to it? And why did none of the dozens of examples I found reference this wget requirement? What am I missing here?

Oh, I would have happily supplied both HTTP and HTTPS examples for your testing. My public file repo supports HTTP downloading. Github.com & raw.githubusercontent.com support downloading files over HTTP too.


So, I’ll try your example code and see where that gets me. I already have a copyFile() function to copy files from one place to another on the SD. Should be able to download to the magical wget directory I didn’t know was needed, then copy to where it belongs. It’ll slow down things for the user a little, as a 1200K file (current size of the .bin) takes about 6-8 seconds to duplicate across the SD…my update routine was originally creating a backup of the WiFiTest.bin file in case of error.

I still can’t get my head around this wget requirement…maybe because I just woke up. haha Where is it coming from and why don’t the lib examples ever show this? They all just download to whatever folder they want (usually SD’s /). Is this wget thing Ringo specific?

1 Like

Yeah, I think it’s because you just woke up!!!

Not in my copy of the firmware or in the master or release 1.0.5 it isn’t. It’s all quite NOT commented out in settingsApp.cpp.

You’ve completely misinterpreted my use of the directory “wget”. I simply chose that for the sample code I listed as an out-of-the-way place to download the file(s) to, so I would know where to look for them. If you want to download it into the WiFiTest directory, that’s fine, you just need to change the directory specified in the SD.remove() and SD.open() calls to whatever you want. There is no “requirement” that the file “must” be downloaded to a directory named “wget”, that’s just what I chose for my example.

Fortunately, the zip file I sort of randomly chose to download is about the same size as your .bin - maybe a little larger - so the download and store time should be about the same. It seems long to me, maybe a minute or more, but then I don’t recall how long it took to download firmware.bin the last time I actually did an update, which was a long time ago because I use a customized firmware that I generate locally.

So given that you can download to wherever you want on the SD card, you won’t have to copy anything.

Hope this clarifies what’s going on. But if you run my examples without changing “/wget/…” to /WiFiTest/…", you will need to create the empty directory or the examples won’t work.