RICO court is basically slang for a court case regarding racketeering charges (ie. violations of the 1970 RICO-Act)
As far as whether the charges are federal, the case listing can be found here. It’s in a D.C. district court, which is a trial court for the federal court system (more info).
Gary J. Byrne has written a book that is negative of the Clintons. In general, the liberal press has denounced his writings as conspiracy theories (example).
Have threats been made against this author for revealing things he shoudn’t have revealed? This lawsuit definitely implies that they have.