[nginx+lua high performance web application development (two): development revie

Recommended for you: Get network issues from WhatsUp Gold. Not end users.

The development of this module, it is in order to solve the actual problem in the project design, thinking .


Preface:


With reference to ngx_lua, Node.js, PHP three for pressure testing.


ngx_lua Time per request: 3.065 [ms]
Node.js Time per request: 3.205 [ms]
PHP Time per request: 5.747 [ms]



Ngx_lualike from a variety of performance test; in the CPU and memory better, at the same time
Ngx_lua surprisingly resource consumption is relatively stable, unlike PHP that fluctuates greatly.
Ngx_lua and Node.js than almost PHP approaching 1 times.
Asynchronous ngx_lua model is single threaded, event driven, work on the same principle and nodejs,
The code or even better than asynchronous callback nodejs to write some.
For maintenance and configuration of NGX the relative ratio of nodejs is more familiar and experienced.


1, The overall design ideas



The complete nginx layer based implementation, nginx to open SSI.
(Server Side Includes: )
Display page with the fastest speed, the use of SSI to read data directly, not return Ajax.
The server to use Lua to read the contents of the database and display.




For applications, use the MySQL partition, the hash according to itme_id.
Set into 1024 partitions, and the library more convenient. Ensure that each table data is not great.
(1024)Is the maximum number of partitions MySQL can set.


Technological process:

Ningx requests /blog/201311/127.html (blog content is static. )




Use the SSI parameter URL which get called Lua comments module



Lua comments module, read the database. (if the pressure is high, can remove the database, cache)


2, The main code



local mysql= require("resty.mysql")
-- Review query.
local function query_mysql(sql)
	local db = mysql:new()
	db:connect{
		host = "127.0.0.1",
		port = 3306,
		database = "test",
		user = "root",
		password = "root"
	}
	local res, err, errno, sqlstate = db:query(sql)
	Using the database connection pool. Stay connected.
	db:set_keepalive(0, 100)
	return res
end

local comment_li = [[

<li id="cmt_" class="row_1">
	<table width=""><tr>
		<td class="portrait">
			<a href="" name="rpl_277051035" class="ShowUserOutline"><img src=" ;></a>
		</td>
		<td class="body">
			<div class="r_title">
				%<b> s floor:%s</b> published in%s
			</div>
			<div class="r_content TextContent">%s</div>
		</td></tr>
	</table>
</li>

]]

-- set the top content.
local comment_html = [[

<div class="Comments" id="userComments">
<h2>
	<a name="comments" href="#" class="more">Back to top</a>
	<a href="#CommentForm" class="more" style="margin-right:10px;color:#ff3;">Comment</a>
	The netizen comments, a total of%s
</h2>
<ul>
%s
</ul>
	<ul class="pager">
		%s
    </ul>
</div>

]]

-- set cycle Li
local page_li = [[
<li class="page %s"><a href="?start=%s#comments">%s</a></li>
]]
Count start limit - input parameters, number, start, paging limits
local function page(count,start,limit)
	--print(count,start,limit)
	local page = math.ceil(count/limit)
	local current_page = math.floor(start/limit) + 1
	Determine if the page is not beyond - page range.
	current_page = math.min(current_page,page)
	current_page = math.max(current_page,1)
	local is_first = ( current_page == 1 )
	local is_last = ( current_page == page )
	-- the number of pages to print, start, end.
	local page_offset = 3
	local start_page = math.max(current_page - page_offset ,1)
	local end_page = math.min(current_page + page_offset ,page)
	local page_html = ""
	if not is_first then
		page_html = page_html..[[<li class="page prev"><a href="?start=]]
		page_html = page_html..(current_page-2)*limit
		page_html = page_html..[[#comments"><</a></li>
		]]
	end
	for i = start_page , end_page  do
		local tmp_current = ""
		if current_page == i then
			tmp_current = "current"
		end
		local tmp_div = string.format(page_li,tmp_current,(i-1)*limit,i)
		page_html = page_html..tmp_div
	end
	if not is_last then
		page_html = page_html..[[<li class="page next"><a href="?start=]]
		page_html = page_html..current_page*limit
		page_html = page_html..[[#comments">></a></li>
		]]
	end
	return page_html
end

- receiving parameters. 
local uri = ngx.var.arg_uri
Using parameter regular filter in URL. 
local year,item_id = string.match(uri,"/blog/(%d+)/(%d+).html")
local start = string.match(uri,"start=(%d+)") or "0"

start = tonumber(start)- strings are converted to digital. 
local limit = 10
- SQL splicing. 

local count_sql = " select count(*) as count from comment where item_id = "..item_id
local count = query_mysql(count_sql)

if count and #count > 0 then
	- assignment of data.
	count  = count[1].count
else
	count = "0"
end
count = tonumber(count)- strings are converted to digital. 

local sql = " select id,uid,content,create_time from comment where item_id = "..item_id.." limit "..start..","..limit
local res = query_mysql(sql)

local comment_lis = ""
for key,val in pairs(res) do
	local id = val["id"]
	local uid = val["uid"]
	local content = val["content"]
	local create_time = val["create_time"]
	- string.
	comment_lis = comment_lis..string.format(comment_li,key,uid,create_time,content)
end

local page_lis = page(count,start,limit)
local html = string.format(comment_html,count,comment_lis,page_lis)
ngx.say(html)

The function of the realization of the code. The follow-up can be split into multiple modules. The generation of paging complex.

Operation: the use of the oschina page style.

Where /blog/201311/127.html is the static HTML page.

Super fast speed.



3, The nginx configuration

Reference resources:




    gzip  on;
    ssi on;
    root /data/comment;
location /blog/list_comment {
default_type 'text/html';
content_by_lua_file '/data/comment/list_comment.lua';
    }
location /blog/save_comment {
default_type 'text/html';
content_by_lua_file '/data/comment/save_comment.lua';
    }




Increasing the SSI code in a static page, so you can think of URI as parameters.
<!--# include virtual="/blog/list_comment?uri=$request_uri" -->

4, Mysql to modify the number of open files

Because open partition, so read the file number must be more.
Ulimit -n view to open the file number. If it is 1024, modify the configuration:


/etc/security/Limits.conf and then restart the system
* soft noproc 65536
* hard noproc 65536
* soft nofile 65536
* hard nofile 65536

And then modify the my.conf configuration file:
vi /etc/my.cnf

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
symbolic-links=0
open_files_limit=65536


Restart the entry into force of the MySQL parameter, see:
mysql> show variables like 'open%';
+------------------+-------+
| Variable_name | Value |
+------------------+-------+
| open_files_limit | 65536 |
+------------------+-------+
1 row in set (0.00 sec)

The database table design, in accordance with the composition area table. Scatter data.


CREATE TABLE `comment` (
  `id` int(11) NOT NULL auto_increment COMMENT 'Primary key',
  `item_id` int(11) NOT NULL COMMENT 'Review ID',
  `uid` int(11) NOT NULL default '0' COMMENT 'User comments',
  `content` text NOT NULL COMMENT 'Content',
  `create_time` datetime NOT NULL  COMMENT 'Create time',
  PRIMARY KEY  (`item_id`,`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8
PARTITION BY HASH (`item_id`)
PARTITIONS 1024;



Insertion of test data:
insert into comment(`item_id`,content,create_time) values(127,CONCAT("conent test ",RAND()*1000),now());

The source code into GitHub.


https://github.com/luapkg/comment/

Continue to improve the review module.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download

Posted by Karen at February 27, 2014 - 3:11 PM