As I've mentioned a few times in this blog, our PCWL series, which comes equipped with mesh Wi-Fi, is something we hope to further develop as a platform for edge computing. Our edge development team is currently working on building the foundation for this. In this blog post, I'd like to provide a concrete example of what developing edge applications -- which are expected to become mainstream -- might look like using the beta version that is currently under development.
To make this experience as immersive as possible, I, who have limited experience in embedded application development, boldly decided to take on the challenge of creating and running a simple edge app on the PCWL-0500. The key to this is utilizing AI. While some knowledge and experience with Linux and embedded systems are necessary, I hope to convey just how easily edge apps can be developed with the help of AI.
Edge apps on the PCWL run in a virtual OS space called a container. This is akin to running a new Linux OS nested within the core OS of the PCWL. The reason we employ such a seemingly cumbersome method is to prevent edge apps from interfering with the operation of the core OS or other edge apps.
For this experiment, I decided to incorporate a simple file caching function as an edge app running on this container within the PCWL. When multiple devices, such as smartphones, tablets, or PCs, within a local area network download a large file from the internet -- like OS update files or 8K high-definition video files -- it results in repeatedly downloading the same file over the local area, which is a waste of bandwidth.
Figure: Overview diagram of the file cache app we developed this time
To mitigate this, the file is first downloaded to local storage within the local area network -- a process known as file caching. By having each device download the file cached in the local storage, the traffic is confined within the local area network, reducing the load on the backbone connection. Additionally, since it does not use the public internet, the download speeds can be significantly faster. For the file caching app being developed this time, large files will be "slowly downloaded" from the internet and stored on a storage device connected to the PCWL's USB port. Thanks to the Samba server (free software that allows access to Linux drives from Windows file systems) that comes standard with the PCWL-0500, the storage can also be accessed by devices connected to the same mesh cluster. This setup allows PCs and other devices within the local area network to access files downloaded to the PCWL from the internet.
The key aspect of "slow downloading"
The key aspect of "slow downloading" is that it allows us to download files with minimal bandwidth consumption in exchange for longer download times. This approach helps prevent a decrease in throughput for other traffic managed by the mesh Wi-Fi system. Files downloaded over time to the USB storage can then be accessed at ultra-fast speeds from PCs within the local area network, without going through the public internet connection. This also prevents the waste of backbone bandwidth that occurs when multiple PCs simultaneously download the same file in parallel.
Let's start with setting up the environment.
First, we install a special firmware customized for edge app development on the PCWL-0500. (Note: This special firmware is only available to developers certified by our company for edge app development. It will not be publicly released, so please be aware.) Next, insert a USB storage device into the USB 3.0 port of the PCWL-0500. For this example, we used a 256GB USB flash drive made by SanDisk.
Access the PCWL-0500's web UI and configure the various settings on the Applications → Samba page.
With this, the Samba server setup is complete. Next, check whether the storage can be correctly accessed from a Windows PC under the same mesh cluster as the PCWL. From the PC's Explorer, select "Map Network Drive" and specify the folder as follows:
"192.168.1.38" is the IPv4 address assigned to the PCWL running the container in my local development environment. After pressing the Finish button, you will be prompted for a user ID and password. Enter the username and password you set on the PCWL's Samba configuration page. If the storage provided by the Samba server is correctly assigned to the PC's Y drive and you can confirm saving and deleting files, the setup is complete.
Next, proceed with preparing the PCWL container side.
After logging into the container, mount the USB storage to a folder. The USB storage is recognized as /dev/sda1 from within the container. Create a folder named /extstorage under the container's /mnt directory and mount it there:
#mount /dev/sda1 /mnt/extstorage
Now, the USB flash drive is accessible from within the container.
Next, we create a program to implement the most important feature: file caching. This is where AI comes into play. I gave the following instructions to ChatGPT:
"Please create a simple file download program that runs on Linux. The language should be Python. Find a test video of about 5GB on the internet and download this video to the /mnt/extstorage directory in the Linux file system. The test video should be in mp4 format. To minimize bandwidth consumption during the download, limit the download speed to 1 Mbps."
ChatGPT provided the following response:
Below is an example of code for creating a simple file download program on Linux using Python. This program downloads a video file from the specified URL and saves it to the specified directory. Additionally, a speed limit of 1 Mbps is set to restrict bandwidth usage.
import requests import time def download_file(url, save_path, max_speed_mbps=1): # 1Mbps = 125000 bytes per second max_speed_bps = max_speed_mbps * 125000 response = requests.get(url, stream=True) total_size = int(response.headers.get('content-length', 0)) block_size = 4096 # ダウンロードするデータのチャンクサイズ if total_size == 0: print("Error: Could not determine file size.") return with open(save_path, 'wb') as file: downloaded = 0 start_time = time.time() for data in response.iter_content(chunk_size=block_size): downloaded += len(data) file.write(data) elapsed_time = time.time() - start_time expected_time = downloaded / max_speed_bps if elapsed_time < expected_time: time.sleep(expected_time - elapsed_time) percent_complete = downloaded / total_size * 100 print(f"Downloaded: {percent_complete:.2f}%") if downloaded >= total_size: print("Download completed successfully!") else: print("Download interrupted or incomplete.") # テスト用の5GB程度のmp4動画URL(例として仮のURLを使用) video_url = "http://example.com/large_test_video.mp" save_directory = "/mnt/extstorage" save_filename = "test_video.mp4" save_path = f"{save_directory}/{save_filename}" download_file(video_url, save_path)
Explanation
Bandwidth Limitation:
The max_speed_mbps parameter is used to limit the bandwidth. In this code, it is restricted to 1 Mbps (1,000,000 bps = 125,000 bytes per second). To control the download speed, the program calculates the amount of data downloaded and the elapsed time, and it uses time.sleep to wait if necessary.
Streaming:
To reduce memory consumption when downloading large files, data is downloaded in a streaming manner (stream=True).
Progress Display:
The progress of the download is displayed as a percentage.
Points to Note:
● You need to specify the URL of the test video to download. Set an appropriate URL in video_url.
● Ensure that the /mnt/extstorage directory exists and has write permissions.
● To run the script, you must have the requests library installed beforehand. If it is not installed, use the following command to install it:.
pip install requests
Unfortunately, ChatGPT couldn't search for a video file of around 5GB. It may be intentionally blocked from introducing external files easily due to security considerations. Therefore, I decided to specify a video file of about 500MB that I found through a Google search. I pasted the URL of this video into the video_url field in the Python code above and executed the code on the container.
Without any issues, test_video.mp4 was downloaded to /mnt/extstorage. I also verified on the Y drive of my Windows PC that the downloaded video file was present and could be played correctly. Since the file size was 500MB and the download speed was limited to 1 Mbps, it took over an hour to complete the download. However, because the bandwidth consumption of the communication line was minimal, there was hardly any noticeable impact on other internet activities (including web meetings) during the download period. The increase in CPU utilization of the PCWL was also less than 1%.
The code, which was 100% created by AI, was error-free and achieved the desired functionality. For those who are unfamiliar with the target programming language, the detailed syntax rules and parameter constraints of that language can often lead to getting stuck, even in short programming tasks. The fact that I could achieve the desired function in one go, without any waste, is nothing short of amazing. I was able to accomplish my goal in about an hour after deciding what to do. And to top it off, I am almost a beginner in Python programming! In the coming era, you will be able to instruct AI in natural language, and AI will handle the programming. This means that humans will be able to focus on thinking about "what they want to achieve with edge computing."
Of course, to have AI generate the correct code, it is important to give it specific and logical instructions. For this, you need a basic understanding of the operating system, communication protocols, and the nature of programming languages. Also, since each AI has different learned knowledge and security policies, it is essential to take the time to interact with your preferred AI to understand its characteristics. Thinking this way, the narrative that "AI will make programmers obsolete," often touted as an AI threat, does not seem like an accurate prediction of the future. Instead, it feels like the skill set required of programmers will evolve with the spread of AI.