MultiCpuSystem
Warning: Any example presented here is provided "as-is" with no support or guarantee of suitability. If you have any further questions about these examples please email the squid-users mailing list.
Contents
![/!\](/National_Library/20160521004321im_/http://wiki.squid-cache.org/wiki/squidtheme/img/alert.png "/!\")
Squid-3.2 has now had experimental multi-process SMP support merged. It is designed to operate with a similar but different configuration to these while reducing much of the complexity of process instance management.
Outline
Squid-3.1 and older do not scale very well to Multi-CPU or Multi-Core systems. Some of its features do help, such as for example DiskDaemon, or COSS, or the ability to delegate parts of the request processing to external helpers such as Authenticators or other auxiliary software. Still Squid remains to this day very bound to a single processing core model. There are plans to eventually make Squid able to effectively use multicore systems, but something may be done already, by using a fine-tuned MultipleInstances setup.
|
This setup has been designed with a recent version of Squid in mind. It has been tested with Squid-3.1, but it should work with Squid-3.0 and Squid-2.7 as-is as well. Squid-2.6 and earlier can be coaxed to work, but it will be harder to setup and maintain. |
It is also geared at expert system-administrators. MultipleInstances is not easy to manage and run, and system integration depends on the specific details of the operating system distribution of choice.
The setup laid out in this configuration example aims at creating on a system multiple running squid processes:
- a 'front-end' process which does
- authentication
- authorization
- logging, delay pools etc.
- in-memory hot-object caching
- load-balancing of the backend processes
- redirection etc.
- a 'back-end' processes farm, whose each does
- disk caching
- do the network heavy lifting
While this setup is expected to increase the general throughput of a multicore system, the benefits are anyways constrained, as the frontend process is still expected to be the bottleneck. Should anyone put this in production, he's encouraged to share the results to help others evaluate the effectiveness of the solution.
Squid Configuration File
For a 2-backends system, there are 5 configuration files to be used. You can click below each file on its filename to download it, no need to copy and paste. The .txt extension an artifact, please remove it.
acl
This file contains the ACL's that are common to all running instances. This allows to change cluster-wide parameters without needing to touch each instance's. Each instance will still have to be reconfigured individually.
1 acl manager proto cache_object
2 acl localhost src 127.0.0.1/32
3 acl to_localhost dst 127.0.0.0/8
4 acl localnet src 192.168.0.0/24
5 acl SSL_ports port 443
6 acl Safe_ports port 80 # http
7 acl Safe_ports port 21 # ftp
8 acl Safe_ports port 443 # https
9 acl Safe_ports port 70 # gopher
10 acl Safe_ports port 210 # wais
11 acl Safe_ports port 1025-65535 # unregistered ports
12 acl Safe_ports port 280 # http-mgmt
13 acl Safe_ports port 488 # gss-http
14 acl Safe_ports port 591 # filemaker
15 acl Safe_ports port 777 # multiling http
16 acl CONNECT method CONNECT
17
18
19
20 cachemgr_passwd somepassword all
common backend parameters
Backends share most of the configuration, it makes sense to also join those
1 #you want the backend to have a small cache_mem
2 cache_mem 4 MB
3
4 refresh_pattern ^ftp: 1440 20% 10080
5 refresh_pattern ^gopher: 1440 0% 1440
6 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
7 refresh_pattern . 0 20% 4320
8
9 shutdown_lifetime 3 second
10 #debug_options all,8
11
12 # add user authentication and similar options
13 http_access deny !Safe_ports
14 http_access deny CONNECT !SSL_ports
15 http_access allow localhost
16 http_access deny all
frontend
1 # acl are shared among instances
2 include /usr/local/etc/lab/common.acl.conf
3
4 http_port 3128
5
6 #add backends
7 cache_peer localhost parent 4001 0 carp login=PASS name=backend-1
8 cache_peer localhost parent 4002 0 carp login=PASS name=backend-2
9
10 #you want the frontend to have a significant cache_mem
11 cache_mem 512 MB
12
13 refresh_pattern ^ftp: 1440 20% 10080
14 refresh_pattern ^gopher: 1440 0% 1440
15 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
16 refresh_pattern . 0 20% 4320
17
18 shutdown_lifetime 3 second
19 #debug_options all,8
20
21 # change /tmp to your own log directory, e.g. /var/log/squid
22 access_log /var/log/squid/frontend.access.log
23 cache_log /var/log/squid/frontend.cache.log
24 pid_filename /var/log/squid/frontend.pid
25
26 # add user authentication and similar options
27 http_access allow manager localhost
28 http_access deny manager
29 http_access deny !Safe_ports
30 http_access deny CONNECT !SSL_ports
31 http_access allow localnet
32 http_access allow localhost
33 http_access deny all
backend 1
1 # acl are shared among instances
2 include /usr/local/etc/lab/common.acl.conf
3
4 http_port 127.0.0.1:4001
5 visible_hostname backend-1
6 unique_hostname backend-1
7 cache_dir aufs /mnt/cache-1 10240 128 128
8
9 # change /var/log/squid to your own log directory
10 access_log /var/log/squid/backend-1.access.log
11 cache_log /var/log/squid/backend-1.cache.log
12 pid_filename /var/log/squid/backend-1.pid
13
14 include /usr/local/etc/lab/common.backend.conf
backend 2
1 # acl are shared among instances
2 include /usr/local/etc/lab/common.acl.conf
3
4 http_port 127.0.0.1:4002
5 visible_hostname backend-2
6 unique_hostname backend-2
7 cache_dir aufs /mnt/cache-2 10240 128 128
8
9 # change /var/log/squid to your own log directory
10 access_log /var/log/squid/backend-2.access.log
11 cache_log /var/log/squid/backend-2.cache.log
12 pid_filename /var/log/squid/backend-2.pid
13
14 include /usr/local/etc/lab/common.backend.conf