Grex has changed drastically since this document was written. It no longer runs under SunOS on a Sun machine, but rather, as of mid-2017, under a modern BSD on an x86 machine. As such, these blocks are no longer used and instead we use the pf packet filter package to accomplish the same thing.
Grex Staff Notes: The Kernel Blocks
It's Grex's policy to allow only validated members to use certain Internet services, like outbound telnet, ftp and IRC. A few other things, like eggdrop, aren't allowed at all, while services like finger and talk are allowed to all users. The reasons for these policies are described in our note on Grex Security Goals. To enforce this, we must block the use of certain services by certain Grex users.
This document explains how these blocks are implemented on Grex. Most of the discussion is in terms of the telnet program, but the same applies to ftp, IRC and other services.
Nope, Not in the Wrapper Scripts
If you look at /usr/local/bin/telnet on Grex, you will see that it is really the following script:
#!/bin/sh
REAL_CLIENTS=/usr/local/grex-scripts/.inet_real
MYNAME=`basename $0`
if [ -r /usr/local/lib/internet.test ]; then
echo " "
else
/usr/local/grex-scripts/no_internet
fi
exec ${REAL_CLIENTS}/${MYNAME} $*
The ftp and irc wrappers scripts are similar. Sometimes people find these and say "Aha! I've found a way to bypass the blocks. The real telnet program is hidden in the .inet_real directory. If I run that instead of the script I'll be able to telnet out."
No such luck. The script is only there to print a nice message explaining to people why the telnet program won't work for them (and to suggest to them that they might want to become members). The blocks are lower down.
Nope, Not in the Telnet Program
So when people find out that the "real" telnet program that is hidden in the .inet_real directory doesn't work either, we often see them downloading their own telnet programs. Apparantly they figure that since some network services, like finger work for them, the blocks that prevent them from telnetting must be in the telnet program.
This isn't a terribly smart thing to try, so it's no wonder that a lot of the people who try it, do it badly. Some download binaries from incompatible systems (clue: only binaries for a SPARC system running SunOS 4.1 have a ghost of a chance of running on Grex). Others download source code and try to compile it (clue: yes, you are allowed to compile programs on Grex, but be aware that Grex is so heavily loaded that large compiles will slow down the system noticably and take a very long time. Grex's staff is almost sure to notice, so be prepared for some attention. If you want to compile something large here, it is always best to talk to the staff first.)
Those people who successfully download or compile their own telnet clients on Grex quickly discover that their's don't work any one wit better than the one in the .inet_real directory. The blocks are not in the telnet program. In fact, the telnet, ftp and irc clients on Grex are among the few programs that we haven't made any changes to. Please don't waste our bandwidth moving other telnet programs here. They really, truely, honestly won't work. The blocks are lower down.
And by the way, the same goes for eggdrop. We don't allow anyone to use eggdrop, so we don't have any version installed here, but if you do succeed in downloading and compiling the thing here without bringing annoyed staff members down on your head, you'll find that it won't work for non-members for exactly the same reasons that telnet and ftp don't work for non-members: the kernel blocks.
Yep, the Blocks are in the Kernel
The kernel is the core of the Unix operating system. It manages all running processes, allocating memory and CPU time to them. It handles all low-level I/O to all the standard devices, including disks, tapes, and serial ports. It also handles some of the higher level abstractions built on top of those devices. On the hard disks, there is the Unix file system and directory tree, with file access permissions enforced by the kernel. For the dial-in serial ports, the kernel takes care of basic terminal emulation, like echoing the characters you type and keeping track of what your backspace character is. And for the serial port that serves as Grex's internet connection, the kernel provides facilities to make internet connections to remote systems and to send and receive packets. That's the part of the system we modified to implement our access restrictions.
If you are used to older microcomputers, you may think a program could just access the serial ports directly, and bypass the kernel. But mainframe computers (and these days, most microcomputers) can run in either supervisor mode or in user mode. The machine code instructions you need to directly access devices only work in supervisor mode and only the kernel runs in supervisor mode. (Even if you are "root", all your programs run in user mode.) Because of this, only the kernel can directly access the hardware devices, and it is not possible to bypass the kernel. This has been the basic concept behind all real security on all real computers for the last thirty years, and it works very, very well.
Where the Kernel Blocks fit in the Kernel
We don't have source for the SunOS kernel, but all modern BSD kernels are structured very much alike (they all derive from 4.2BSD, which was developed at the Univeristy of California, Berkeley with public funds, and is thus public domain). Thus, if you look at the publically available FreeBSD, NetBSD or OpenBSD kernels gives you'll get a good idea of how the SunOS kernel works (we also disassembled very small portions of the SunOS kernel to confirm that it hadn't changed too much). There are also several textbooks describing in detail how all this works (see, for example, Leffler, McKusick, Karels and Quarterman, The Design and Implementation of the 4.3BSD Operating System, Addison Wesley: 1989).
There are several layers to the implementation of networking in BSD kernels. We are specifically interested in the layers implementing the basic Internet UDP and TCP protocols. UDP is a very simple protocol for sending individual packets (which may or may not arrive) over the net. It is most commonly used for communicating with domain name servers and establishing talk connections. TCP is a much more complex and much more widely used protocol that establishes connections between pairs of machines, over which streams of information may be reliably sent.
The kernel routines handling TCP and UDP communications activity are organized by the kinds of events they respond to:
- udp_usrreq() and tcp_usrreq()
- handle all requests from user programs, such as to send data, fetch data from a receive buffer, or (for TCP only) to open or close a connection.
- udp_input() and tcp_input()
- handle the arrival of a packet from another system.
- tcp_timer()
- handle the expiration of a timer (TCP only).
Before any kind of UDP or TCP packets can be sent or received, your application program must ask the kernel to create a socket in the Internet domain, and it must specify a network address and port number to send or connect to. (The network address selects the remote machine, and the port number selects the service on that machine.) There are several ways that your application program can specify the remote address and port, including the connect(), bind(), sendto(), and sendmsg() kernel calls, but all of these methods result in calls to either udp_usrreq() or tcp_usrreq(), and in either case, a routine named in_pcbconnect() is called to actually save the information.
It is around the in_pcbconnect() routine that we place our wrappers. The wrappers cause in_pcbconnect() to appear to fail if the particular connection being requested is not permitted by Grex policy.
Sun doesn't distribute source code for their kernel, but they do distribute a kernel kit, consisting of hundreds of already compiled ".o" object files, which can be linked together to form a runable kernel. So we were able to make our changes by making some small edits to one of the object modules, causing it to call our routines instead of Sun's, and adding in our own new module.
What we did was write two new routines, called in_TCPconnect() and in_UCPconnect(), which do the needed checks, and then either exit with an error code or call the in_pcbconnect() routine. We then manually editted the symbol tables in the SunOS kernel object files, replacing calls to in_pcbconnect() with either in_TCPconnect() or in_UDPconnect() (the names are the same lengths, so this is an easy edit). The call to in_pcbconnect() in tcp_usrreq was replaced with a call to in_TCPconnect(), and the two calls in upd_usrreq were replaced with calls to in_UDPconnect(). That way, after we relinked the kernel, all calls to in_pcbconnect() had to pass our tests before being processed.
Most of the work on this was done by Grex staff members Marcus Watts and Steve Weiss.
The TCP wrapper
The TCP wrapper, in_TCPconnect() allows the connection to be made only if one of the following is true:
- The user has a uid number less than 1000. On Grex, this allows system processes, including root processes, and staff, to start TCP connections.
- The connection being established is to "localhost", or to a host on Grex's subnet. We are trying to restrict access to the outside Internet, not to our own systems.
- The connection is to a port number on the following list:
43 whois 53 dns 70 gopher 79 finger 80 http 113 ident 517 talk 518 ntalk Thus, all users, even non-members, can have access to these standard Internet services.
- The user is a member of group 95, the "internet" group. This allows the people who are listed in the "internet" group in the /etc/group file to have access to all TCP services.
Here's the actual code (actually, it's a slightly out-of-date version, with Grex's old IP address). Note that if all goes well, we call the real routine, in_pcbconnect(). Otherwise, we log an error and return the access error code.
in_TCPconnect(inp, nam)
struct inpcb *inp;
struct mbuf *nam;
{
int i;
struct proc *p= u.u_procp;
ushort uid = p->p_cred->cr_uid;
int *grps = p->p_cred->cr_groups;
struct sockaddr_in *sin = mtod(nam, struct sockaddr_in *);
long saddr;
if (nam->m_len < sizeof *sin) goto Skip;
saddr= ntohl(sin->sin_addr.s_addr); /* for platform completeness */
if ( (uid >= 1000) && u.u_ruid
&& ((saddr & 0xFFFFFF00) != (127 << 24))
&& ((saddr & 0xFFFFFF00) != ((152 << 24) + (160 << 16) + (30 << 8) ))
&& ((saddr & 0xFFFFFFE0) != ((198 << 24) + (87 << 16) + (20 << 8) + 32)) && (sin->sin_port != 43) /* whois */
&& (sin->sin_port != 53) /* DNS (dig) */
&& (sin->sin_port != 70) /* gopher */
&& (sin->sin_port != 79) /* finger */
&& (sin->sin_port != 80) /* http */
&& (sin->sin_port != 113) /* ident */
&& (sin->sin_port != 517) /* talk */
&& (sin->sin_port != 518) /* ntalk */
)
/* if all of the above *and* conditions are true, the user must be a
member */
{
/* Test for membership: This assumes that group "internet" == 95 */
for(i = 0;(i < NGROUPS) && (grps[i] != 0) && (grps[i] != 95);i++)
;
if ( (i == NGROUPS) || (grps[i] == 0) )
{
log(LOG_INFO,
"b_TCP: %d/<%s> denied %d.%d.%d.%d:%d, %d/%d\n",
p->p_pid, u.u_comm,
(unsigned char)(saddr>>24),
(unsigned char)(saddr>>16),
(unsigned char)(saddr>>8),
(unsigned char)(saddr),
sin->sin_port, u.u_ruid, u.u_rgid);
return EACCES;
}
}
Skip:
return in_pcbconnect(inp, nam);
}
The UDP wrapper
The UDP wrapper, in_UDPconnect() is very similar. It makes the requested connection under the any of the following conditions:
- The user has a uid number less than 1000.
- The connection being established is to "localhost", or to a host on Grex's subnet.
- The connection is to a port number on the following list:
53 dns 517 talk 518 ntalk - The user is a member of group 95, the "internet" group.
Here's the actual code for the in_UDPconnect() wrapper.
in_UDPconnect(inp, nam)
struct inpcb *inp;
struct mbuf *nam;
{
int i;
struct proc *p= u.u_procp;
ushort uid = p->p_cred->cr_uid;
int *grps = p->p_cred->cr_groups;
struct sockaddr_in *sin = mtod(nam, struct sockaddr_in *);
long saddr;
if (nam->m_len < sizeof *sin) goto Skip;
saddr= ntohl(sin->sin_addr.s_addr); /* for platform completeness */
if ( (uid >= 1000)
&& ((saddr & 0xFFFFFF00) != (127 << 24))
&& ((saddr & 0xFFFFFF00) != ((152 << 24) + (160 << 16) + (30 << 8) ))
&& ((saddr & 0xFFFFFFE0) != ((198 << 24) + (87 < 16) + (20 << 8) + 32))
&& (sin->sin_port != 53) /* dns */
&& (sin->sin_port != 517) /* talk */
&& (sin->sin_port != 518) /* ntalk */
)
/* if all of the above *and* conditions are true, the user must be a
member */
{
/* Test for membership: This assumes that group "internet" == 95 */
for(i = 0;(i < NGROUPS) && (grps[i] != 0) && (grps[i] != 95);i++)
;
if ( (i == NGROUPS) || (grps[i] == 0) )
{
log(LOG_INFO,
"b_UDP: %d/<%s> denied %d.%d.%d.%d:%d, %d/%d\n",
p->p_pid, u.u_comm,
(unsigned char)(saddr>>24),
(unsigned char)(saddr>>16),
(unsigned char)(saddr>>8),
(unsigned char)(saddr),
sin->sin_port, u.u_ruid, u.u_rgid);
return EACCES;
}
}
Skip:
return in_pcbconnect(inp, nam);
}
Conclusion
There is admittedly something fundamentally klunky about hardwiring Grex's administrative policies into our operating system's kernel. It means that if we want to change the services we offer (or change our IP address) we need to rebuild the kernel. But on a system where we give all users full access to Unix development tools, that's where it has to be.
Still looking for an easy way to get around our blocks? Click here.