As planned, I have completed the caching plugin. But as the pencils down date is tomorrow, I’m still trying to improve the performance and clean up the code.
This week (especially today with the help of Eduardo), I have made a lot of changes to the plugin code.
However, according to the initial plan, I was also supposed to get a user interface up and running by the end of GSoC. Although I have written the code for the same, it is not in working condition yet. There still needs to be some fixing and changing to be done. I hope to improve the code more before the pencils down timing tomorrow. If I am not able to complete it entirely I will continue to work on this post GSoC and finish it as was planned initially.
Like everybody else, I’ve also learnt a lot during the internship. Thanks to Eduardo for being patient with me.
Last week I was mainly working on changing the method of deletion of files from the cache from using the min heap to using time as a basis to remove the expired files, to see if there is any improvement in the performance (Not much to “write about”.). However, this has been temporarily stopped (taking into account a suggestion from Eduardo). It will be resumed later. I also need to add a button to the user interface, to purge all the documents in the cache. I will also try to incorporate the suggestion provided by Eduardo, to use mk_file_get_info to improve the performance.
I am doing testing on the code and surprisingly a few errors have popped up – some grave errors at that ! I’m trying to fix them and finish off the documentation in this week.
This week I spent most of my time working on getting the real time chart display to get to work – solving the errors that I encountered, mainly due to not including the necessary source files for the charts. I used flot charts to draw the real time graph that has number of requests served by the server and the time on X and Y axes respectively. I am testing if the display is working correctly. I also added code for JSON output in the main file for the cache plugin.
After this, I will need to tie up some loose ends in the code and do documentation. I expect to finish the project by this Friday and post the code for reviews and comments. I will post more about this then !
This week(end) I was working on getting the cache statistics and live displays to work. I have accomplished part of the work. Part of it still remains and I hope to complete it by tomorrow.
Cache Statistics :
It has to be determined how much of requests each thread is handling. The new functions added are :
- cache_process_init – to initialize the cache statistics when the cache plugin is running and when the user interface is started.
- cache_thread_init – to initialize the cache statistics for each thread of the server that is running.
- cache_update – to update the statistics of the plugin, adding up the request per second that are currently running.
- cache_stats_new – each time a new request is assigned to a thread, the number of requests handled by the specific thread is incremented.
Finding mime type of requested file :
Making use of mime type module of Monkey server, I also wrote code to pass the mime type of the requested file to the response header (something that I had not done before). The way to obtain the mimetype of the file requested, is to find out the extension of the file from the url of the request. This is then set to the mime type of the request header in the cache.c file. This is exactly how the server has found the content type of the requested file. This could be used in the cache plugin too. Using the mimetype_find () function present in the server’s source code file – mimetype.c, the mime type is found, by passing the file name.
User interface development
I spent most of my time on figuring out how to include graphs in the user interface and make them display live information. This link : http://stackoverflow.com/questions/22685743/how-to-populate-real-time-data-into-a-twitter-bootstrap-admin-dashboard-theme was useful. I tried to use Flot charts in the user interface. I also had to include jquery.flot.js and jquery.js files to in the dependencies. I figured how to do that.
Adherence to timeline :
According to the timeline, I need to finish the user interface and start testing from the next day. I hope to complete this task by tomorrow and start testing from the next day or at least by Wednesday.
Code : https://github.com/tssavita/cache-plugin-implementation
This week, I managed to fix whatever the issue was, that was causing a decrease in the performance of the caching plugin. Also apart from that I designed the UI (that is to be designed using Twitter bootstrap and Angular JS) – what all should be printed on it, such as
- a graphical display of the requests per second.
- cache clearing
- cache hits and misses
I started out with writing code to calculate these data when the plugin is running.
I also started out with changing my cache from Least Commonly Used to Least Recently Used, to see if it improves the performance of the plugin, but however discontinued this job to prioritize other remaining tasks.
During the next week,
- After testing the plugin for various corner cases, I plan to change my cache type from LCU to LRU
- I plan on completing code for calculation of the statistics data, and integrating them with the user interface.
Reference and special mention :
This week I was trying to find out what is the cause of the error that degrades the performance of Monkey. With the help of Eduardo (my mentor), I found out a few errors.
Web server benchmarking :
Server side profiling : I used a new tool of the name kcachegrind, which is a variation of valgrind, that helps find out how many times each function in the source code has been called and which function takes the maximum processing time.
Two simple things that I discovered that I was doing wrong are :
- For the purpose of debugging, I had been printing messages to the standard out, from the various functions of the caching plugin. I was also configuring the server with the trace mode on. This greatly slowed down the performance of the server.
- Also, at the end of stage_30 of the plugin, inspite of the request being successfully being handled by the plugin, it returned MK_PLUGIN_RET_NOT_ME, which meant that it would be handled by monkey core. This also affected the performance. I’m still working on fixing this issue and returning MK_PLUGIN_END at some point so that the request would be completely processed.
This blog post is regarding what I did during the week 7 of my GSoC internship. Due to unexpected emergencies, I could not access my laptop during the past few days because of which I could not send the update for the week on time.
I got some errors while integrating, such as :
bin/monkey: symbol lookup error: <path to monkey source code> /plugins/cache/monkey-cache.so: undefined symbol: je_malloc
I had to enable the –malloc-libc to get it fixed. But with no result. I used PLUGIN_TRACE to find out location of errors. I then tried to play around with the header files in the dependencies folder by including them in the necessary files this seemed to workout for me !
The next issue that I faced was that of a segmentation fault. I again used PLUGIN_TRACE to debug and find out the location and it turned out that it was being caused due to a null point exception.
These were a few among a list of other errors that I got.
I then did a basic integration of the caching plugin with the server – just an extra feature to look up files from the cache. I tried testing the performance of the server with and without the caching plugin, with the help of weighttp (a lightweight web server performance benchmarking tool) and found that I have done some grave error because of which the performance of the server deteriorates even more, when I run the server with the caching plugin (which is not supposed to happen !). I am looking into it right now.
Parallely I also tried out some basic tutorials for creation of GUI using Twitter bootstrap and Angular JS. This was the easiest part. But since as of now, the caching plugin is not yet ready, I will be concentrating on it till I get it fixed.
I hope to get it ready and working soon !
Link to code repo : https://github.com/tssavita/cache-plugin-implementation