Refactor/service framework method shadowing

[renatomaia]

Issues Addressed

• Propagate errors from services through the service framework.
• Provide better support for services that do not perform periodic batch processing (tick-based), like the ones that provide a HTTP interface (e.g. Advancer, JSON-RPC) or consume external events like a WebSocket (i.e. EVM Reader).
• Avoid multiple stages during service shutdown (Running, Stopping, and Stopped).
• Avoid method shadowing of service framework methods.

Still Open Issues (left for a future PR)

• There are two ways to stop services: either calling Stop(force bool) or by cancelling the context explicitly provided to be used by the service. The former is used to synchronize the shutdown of multiple services running together.
• Extend the support for services oriented to periodic batch processing (tick-based) to also support for services oriented to provide HTTP interface, and also services that consume external events like a WebSocket interface.
• Provide better support to organize the code as independent services running under a single process. Examples include the Node running as a single process, or the Advancer HTTP API running with and Inspector, and Telemetry API in a single process.

Overview of Changes

Package pkg/service provides the service runtime used across the node. It owns lifecycle concerns such as logging, signal handling, telemetry, and shutdown coordination.

The solution seems to try to follow the Template Method design pattern where a base class implements methods that provide those basic features (template methods defined by interface IService) but also calls additional methods that are provided by the subclass to implement some service specific behavior (abstract methods defined by interface ServiceImpl).

package service

type ServiceImpl interface {
  Alive() bool
  Ready() bool
  Reload() []error
  Stop(bool) []error

  Tick() []error
}

type IService interface {
  Alive() bool
  Ready() bool
  Reload() []error
  Stop(bool) []error

  Serve() error
  String() string
}

type Service struct {
  Name string
  ...
  Impl ServiceImpl
}
func (s *Service) Alive() bool { return s.Impl.Alive() }
func (s *Service) Ready() bool { return s.Impl.Ready() }
func (s *Service) Reload() []error { ... return s.Impl.Reload() }
func (s *Service) Stop(force bool) []error { ... return s.Impl.Stop(force) }
func (s *Service) Serve() []error { return for { s.Impl.Tick() ... } }
func (s *Service) String() string { return s.Name }

////////////////////////////////////////////////////////////////////////////////

package custom_service

type Service struct {
  service.Service
  ...
}
func Create() *Service {
  s := &Service{Name: "Custom Service"}
  s.Impl = s  /* same as 's.Service.Impl = s' */
  return s
}
func (s *Service) Alive() bool { return true }
func (s *Service) Ready() bool { return true }
func (s *Service) Reload() []error { return nil }
func (s *Service) Stop(force bool) []error { return nil }
func (s *Service) Tick() []error { ... return nil }

In Go, it is common to use struct embedding to emulate class inheritance of other languages. However, Go does not provide a standard way for a embedded struct (base class) like service.Service to access its embedding struct (subclass) custom_service.Service. Therefore it is always necessary to explicitly provide a reference that allows the template method to access the actual implementation of the abstract method it needs to call (see the description of one approach here). This explicit reference is provided as field service.Service.Impl.

Such approach for Template Methods using struct embedding with explicit auto-reference has the following advantages:

1. Since both template and abstract method implementations are accessible in the embedding struct (subclass), abstract methods can easily access the API of the base class. For instance, access the Logger from the base class or call Cancel to terminate the service's context and cause the service to stop (it would be better to be able to call Stop but it is shadowed due to a particular problem of the current implementation).
2. By using the instance to the embedding struct (subclass) as the service object implementing IService for the clients, the embedding struct (subclass) can override any of its public method by shadowing the "inherited" implementation. For instance, some services redefine the Server method to do continuous processing instead of the tick-based periodic processing supported by the service framework.

Problems

1. Using the same method name for a template method and the abstract method that it invokes causes some confusion when adopting the approach of struct embedding because the abstract method shadows the template method. Therefore clients using the objects returned by factory function custom_service.Create() will execute the stripped down abstract method instead of the full-featured template method. In principle, we could solve this by returning the embedded service.Service instead of the custom_service.Service object, but this would prevent shadowing altogether, which is an issue as described by the next problem.

2. Since service.Service only supports services that perform smalls processing batches periodically (ticks), some custom services define an implementation of Serve that shadows the one from service.Service to start Goroutines for continuous tasks. Therefore, we currently must return custom_service.Service to allow to shadowing of some methods.

Solution

Adopt different method names for abstract methods to avoid unintentional method shadowing. It also makes clear that calling Stop on custom_service.Service will not execute custom_service.Service.Stop as the former will be named differently.

Unlike methods Reload, Stop, and Serve, methods Alive and Ready do not implement a common basic behavior and only delegate entirely to the custom service. For this reason, we will not implement them as a pair of template and abstract methods, and just let the concrete services to shadow them when necessary to define a custom behavior.

We will also implement a standard behavior for most abstract methods which are currently implemented by most services.

We will also provide two alternative service templates:

• ServiceTemplate for services that do continuous processing.
• TickServiceTemplate for services that do processing in periodic batches (tick-based).

Finally, since the impl self-reference is a specific hack necessary in Go to implement the Template Method design pattern, we removed this field from the service initialization structure (current ConfigInfo) to avoid making them too visible in the code, and try to confine this setup of the self-references to the factory functions that create and initialize services.

package service

// Public interface with methods to manipulate the service.
type IService interface {
  Alive() bool
  Ready() bool
  Reload() []error
  Stop(bool) []error
  Serve() error
  String() string
}

/*
 * Template for services that do continuous processing.
 */

// Internal interface with abstract methods called by the template methods.
// These methods are not part of the public service API.
type LifecycleImpl interface {
  OnReload() []error
  OnStop(bool) []error
  DoServe() []error
}

type ServiceTemplate struct {
  lifecycle LifecycleImpl
  Name string
  ...
}

type ServiceConfigs struct {
  Name string
  ...
}

func InitTemplate(
  cfg *ServiceConfigs,
  tmpl *ServiceTemplate,
  impl LifecycleImpl,
) {
  /* assume impl.ServiceTemplate == tmpl */
  tmpl.lifecycle = impl  // set explicit auto-reference used by template methods
  tmpl.Name = cfg.Name
  ...
}

// Default implementation of some abstract methods (except `Tick`).
// Remove them to force concrete services to provide implementation for them.
func (s *ServiceTemplate) OnReload() []error { return nil }
func (s *ServiceTemplate) OnStop(bool) []error { return nil }
func (s *ServiceTemplate) Alive() bool { return true }
func (s *ServiceTemplate) Ready() bool { return true }
func (s *ServiceTemplate) String() string { return s.Name }

// Template methods implementing basic behavior.
func (s *ServiceTemplate) Reload() []error { ... return s.lifecycle.OnReload() }
func (s *ServiceTemplate) Stop(force bool) []error { ... return s.lifecycle.OnStop(force) }
func (s *ServiceTemplate) Serve() []error { ... return s.lifecycle.DoServe() }

/*
 * Alternative template that implements the tick-based processing.
 */

// Internal interface with abstract methods called by the template methods.
// These methods are not part of the public service API.
type TickImpl interface {
  Tick() []error
}

type TickServiceTemplate struct {
  ServiceTemplate
  tickImpl TickImpl
  ticker   *time.Ticker
  ...
}

type TickServiceConfigs struct {
  ServiceConfigs
  PollInterval     time.Duration
  EnableReschedule bool
}

func InitTickTemplate(
  cfg *TickServiceConfigs,
  tmpl *TickServiceTemplate,
  svcImpl LifecycleImpl,
  tickImpl TickImpl,
) {
  /* assume tickImpl.TickServiceTemplate == t */
  tmpl.tickImpl = tickImpl  // set explicit auto-reference used by template methods
  InitTemplate(cfg.ServiceConfigs, tmpl.ServiceTemplate, svcImpl)
  ...
}

// Internal API with methods provided for use in implementation of concrete services.
func (s *TickServiceTemplate) SignalReschedule() { ... }
func (s *TickServiceTemplate) DrainReschedule() bool { ... }  // used only for unit testing

// Default implementation of abstract method used by `ServiceTemplate`.
func (s *TickServiceTemplate) DoServe() []error { ... for { ... s.impl.Tick() ... } ... }

////////////////////////////////////////////////////////////////////////////////

package custom_service

type CustomService struct {
  service.TickServiceTemplate
  ...
}

type CustomConfigs struct {
  service.TickServiceConfigs
  ...
}

func NewCustomService(cfg *CustomConfigs) service.IService {
  s := &CustomService{}
  service.InitTickTemplate(cfg.TickServiceConfigs, s.TickServiceTemplate, s, s)
  return s
}

// Override public methods by shadowing.
func (s *CustomService) Alive() bool { ... }
func (s *CustomService) Ready() bool { ... }

// Override default implementation of abstract methods used by the `ServiceTemplate`.
func (s *CustomService) OnReload() []error { ... }
func (s *CustomService) OnStop(force bool) []error { ... }
func (s *CustomService) DoServe() []error { ... return s.TickServiceTemplate.DoServe() }

// Implement abstract methods used by the `TickServiceTemplate`.
func (s *CustomService) Tick() []error { ... }