Interactive judge

The interactive judge gets the input and the means of communication with the contestant's solution. It should report whether the contestant's solution is correct.

There is currently only one interactive judge_type supported:

CMS-communication judge

See CMS documentation for more. However, only one user process is supported.

The solution and the judge run simultaneously. They communicate via FIFO (named pipes).

The judge is run:

./judge <FIFO from solution> <FIFO to solution> < input

FIFOs point to the solution's stdout and stdin respectively.

The judge should print a relative number of points (a float between 0.0 and 1.0) to its stdout as a single line. To its stderr it should write a single-line message to the contestant. Unlike what the CMS documentation specifies, the files should be single-line only. There will be a warning otherwise.

Example cms-communication judge

For a binary search task the judge may look like this:

#!/usr/bin/env python3
import sys
from typing import NoReturn

MAX_QUERIES = 1000


def verdict(pts, msg=None) -> NoReturn:
    print(pts)

    if msg is None:
        if pts <= 0:
            msg = "translate:wrong"
        elif pts >= 1:
            msg = "translate:success"
        else:
            msg = "translate:partial"

    print(msg, file=sys.stderr)
    exit(0)


def reject(msg=None) -> NoReturn:
    verdict(0, msg)


def protocol_violation() -> NoReturn:
    verdict(0, "Protocol violation")


_, recv_name, send_name = sys.argv
n, x = map(int, input().split())
values = list(map(int, input().split()))
correct_answer = values.index(x) if x in values else -1

with open(send_name, "w") as send_pipe:
    with open(recv_name) as recv_pipe:

        def send(*x: int) -> None:
            try:
                print(*x, file=send_pipe)
                send_pipe.flush()
            except BrokenPipeError:
                protocol_violation()

        send(n, x)
        queries = 0
        while queries <= MAX_QUERIES:
            line = recv_pipe.readline().split()
            try:
                query_type, qv = line
                query_value = int(qv)
            except ValueError:
                protocol_violation()

            if query_type == "?":
                queries += 1
                if 0 <= query_value < n:
                    send(values[query_value])
                else:
                    send(-1)
                    protocol_violation()

            elif query_type == "!":
                if query_value == correct_answer:
                    verdict(1)
                else:
                    reject()  # Wrong answer

            else:
                protocol_violation()

        send(-1)
        reject()  # Too many queries