Rule DSL

My work focused also on the implementation of the hint system, which provides players with contextual feedback. The system's core is a declarative Domain-Specific Language (DSL) designed to define analysis rules in a type-safe and expressive way. The process, orchestrated by the HintEngine, analyzes the player's history of CaseKnowledgeGraph states, computes metrics, identifies trends, and evaluates these rules to determine if a hint should be generated by the LLM.

The Rule DSL

The DSL, located in the model.hint.RuleDSL package, was designed to be highly readable, allowing for the definition of complex logic in a near-natural language format.

The entry point is the when function, which wraps a metric function (e.g., density) into a MetricExpr. This expression is then compared to a Trend (e.g., Increasing, Stable) using the == operator, which creates a MetricCheck. MetricChecks are the fundamental building blocks of a rule's condition and can be composed using logical and and or operators. The hence operator completes the rule by associating the condition with a HintKind (Helpful or Misleading).

final case class MetricCheck[T](eval: List[T] => Boolean):
  infix def and(other: MetricCheck[T]): MetricCheck[T] =
    MetricCheck(obj => eval(obj) && other.eval(obj))

  infix def or(other: MetricCheck[T]): MetricCheck[T] =
    MetricCheck(obj => eval(obj) || other.eval(obj))

  infix def hence(hint: HintKind): Rule[T] = Rule[T](this, hint)

final case class MetricExpr[T](compute: T => MetricValue):
  infix def ==(trend: Trend)(using analyzer: TrendAnalyzer): MetricCheck[T] =
    MetricCheck[T] { objs =>
      val values = objs.map(compute)
      analyzer.analyze(values) == trend
    }

def when[T](metric: T => MetricValue): MetricExpr[T] = MetricExpr[T](metric)

final case class Rule[T](condition: MetricCheck[T], hint: HintKind)

This design results in rules that are both powerful and easy to understand. A simple rule to generate a Helpful hint when the graph's density is stable is defined concisely in model.hint.HintEngine.scala:

given stableDensity: Rule[BaseOrientedGraph] = when(density) == Stable hence Helpful

More complex rules can be created by combining multiple checks. For instance, a rule to generate a Misleading hint if the player is steadily increasing their graph's coverage but the density is not improving can be expressed just as clearly:

when(coverageAgainst(reference)) == Increasing and 
when(density) == Stable hence Misleading

Testing the DSL

The DSL's logic was thoroughly verified in RuleDSLTest.scala to ensure that all components function as expected. The tests cover individual metric checks, the logical combinators, and the final evaluation of complete rules.

A single condition is tested by creating a MetricCheck and evaluating it against a controlled history.

"MetricCheck" should "evaluate to true when condition matches" in:
  Given("a history of graphs with incrementing density")
  val history = List(
    graph.withNodes(1, 2, 3),
    graph.withNodes(1, 2, 3).withEdge(1, "link1", 2),
    graph.withNodes(1, 2, 3).withEdge(1, "link1", 2).withEdge(2, "link2", 3)
  )

  When("checking if density is increasing")
  val check = when(density) == Increasing

  Then("the check should evaluate to true")
  check.eval(history) shouldBe true

The logical combinators are tested by creating compound checks and asserting their boolean logic.

"MetricCheck combinators" should "work with 'and' operator" in:
  Given("a history with increasing coverage and worsening density")
  val reference = graph.withNodes(1, 2, 3)
  val history = List(
    graph.withNodes(1, 2).withEdge(1, "link1", 2),
    graph.withNodes(1, 2),
    graph.withNodes(1, 2, 3)
  )
  
  When("checking both conditions with 'and'")
  val check = when(coverageAgainst(reference)) == Increasing and when(density) == Worsening

  Then("the combined check should evaluate to true")
  check.eval(history) shouldBe true

Finally, the end-to-end evaluation of a complete Rule is verified by using the HintEngine to ensure it returns the correct HintKind when a rule's condition is met.

"HintEngine.evaluate" should "return Some(hint) when condition matches" in:
  Given("a rule and matching history")
  val reference = graph.withNodes(1, 2, 3)
  val history = List(
    graph.withNodes(1),
    graph.withNodes(1, 2),
    graph.withNodes(1, 2, 3)
  )
  val rule = when(coverageAgainst(reference)) == Increasing hence Misleading

  When("evaluating the rule")
  val result = HintEngine.evaluate(history)(using rule)

  Then("it should return the hint")
  result shouldBe Some(Misleading)