Koryu literally means "old flow" and is used in Japanese to refer to old styles, schools, or traditions (not necessarily only in the martial arts). The simplest English translation of this term is "classical." Bujutsu is one of several Japanese terms meaning martial arts or techniques (I won't get into the whole jutsu/do issue here; there are differences between "do" arts and "jutsu" arts but they are not as large as many people imagine, in my opinion). So, putting the two together, you get "classical martial arts" (of Japan).
This leads to the next question: What precisely is a Japanese classical martial art? Unfortunately, it is impossible to provide a precise definition. Some folks use the date of 1876, when the act prohibiting the wearing of swords (Haitorei) was promulgated, as a dividing line between classical and modern martial arts. Others prefer to use the term "koryu bujutsu" only for those arts developed for actual use on the battlefield and thus reserve the term for sixteenth century and earlier traditions.
Classical traditions (however you date them) do have several defining characteristics. They were developed by and for the bushi, or warrior class, and they have some sort of lineage that runs back through each headmaster or menkyo kaiden (or equivalent) to the founder of the tradition. This is the "stream" of the tradition; although there may be branches (in some traditions each generation was expected to found their own "sub-tradition"), in general the structure is one of a single "flow" from one (usually) head instructor through students to the next generation. The student-teacher relationship is central to transmission of techniques, and these techniques were initially designed to ensure victory on the battlefield. Systems that don't exhibit all these characteristics are quite likely not koryu.
In actual practice, both in Japan and in the West, koryu bujutsu (pronounced "core-you boo-jut-su"--never "core-roo" please!) is used to refer rather loosely to the older martial arts of the bushi or samurai. Usually the emphasis is on distinguishing between the classical systems and the gendai budo (modern martial arts), such as judo, kendo, and aikido. These modern arts all have some connection to the older fighting systems of the warrior class: judo, for example, derives primarily from Tenjin Shinyo-ryu and Kito-ryu jujutsu; kendo comes from a number of different old-style, or koryu, kenjutsu traditions. Modern arts were developed after the disbanding of the warrior class, can be taught to large groups, and are designed to educate and develop the human being as a whole.
The koryu (as we sometimes say for short), on the other hand, were primarily arts created by and for the warrior class of Japan's feudal period. A few traditions still exist that were actually used on the battlefields of pre-Tokugawa Japan, and in these systems effectiveness of the killing technique is still paramount. Other traditions were developed during the peace of the Tokugawa shogunate to help warriors-turned-bureaucrats maintain some level of fighting skill; still others were meant from the start to be "disciplines," with more emphasis on spiritual benefits than on technical skills useful in fighting. Thus, the koryu exist (both today and in the past) along a continuum of purposes, much like that of modern American martial arts. For the most part, however, the techniques of the koryu still retain an element of danger; protective gear is typically not used. Safety is less important than efficacy; though wooden weapons are usually used in place of live steel blades, these can still do considerable damage if an error is made, and one learns to function out at the edge.
What distinguishes the koryu most clearly from the modern disciplines is the social structure of the ryu--which is a very particular and peculiar Japanese construct that cannot be faked (see the review of Geisha, for some more information). Though there are some made-in-the-USA ersatz koryu, who parade around in rigid hierarchies with elaborate Japanese trappings, anyone who has functioned within a genuine koryu can spot the fraud. In the true koryu traditions, culture and technique are part of a cohesive whole that includes the headmaster, traditional licenses, and a unique code of behavior. While there's no question that it is possible to learn/teach just the techniques of a koryu in isolation from the rest (and there are those who are doing this--the open seminar format, so popular in the West is a good example), the result is, in my opinion, a mere parody of the rich wealth, both technical and spiritual, found in these ancient traditions.
What this means, for practical purposes, is that if you cannot go and spend a decade or more training in Japan, you MUST seek out someone who has. To participate fully in the koryu you must be in direct contact, through your teacher if needs be, with the headmaster (soke) or other fully licensed instructor (menkyo kaiden or equivalent). The relationship between student and teacher is the core of both the social structure and technical transmission. Without it, you might as well be learning tiddly-winks (no offense to serious tiddly-winkers intended)! Each student is taught individually, and not everyone learns the same things in the same sequence or at the same pace. Instead of the dan grades characteristic of the modern arts, the teacher issues a traditional license--and this process is not standardized or codified. When the teacher judges the student to be ready, the license is granted.
This is merely the briefest of overviews. One important fact to keep in mind when studying the koryu is that it is impossible to make entirely accurate generalizations. Each ryu is a unique and individual entity with a distinctive identity. The name of the highest license in one art may be the lowest level of license in another; some use entirely unique terms for their licenses. Some traditions function under the control of a single headmaster; others are divided into several lines under the guidance of fully licensed instructors. Some traditions have no headmaster at all. The best source for information on any given ryu is a member in good standing, for many of the details of a particular ryu's culture are never written down.
For our own training group, we expect prospective students to do some background reading before they may begin training. If you are seriously interested in training in the koryu, I'd highly recommend that you read the following BEFORE you begin to look for an instructor.