237 stories
·
2 followers

Pastrami Benedict

1 Comment
Read the whole story
kbrint
8 days ago
reply
Top notch idea.
Share this story
Delete

Challengers

5 Comments and 8 Shares
Use your mouse or fingers to pan + zoom. To edit the map, submit your ballot on November 6th.
Read the whole story
kbrint
9 days ago
reply
To edit the map, VOTE!
Share this story
Delete
3 public comments
deezil
11 days ago
reply
This is so fantastic.
Louisville, Kentucky
alt_text_at_your_service
12 days ago
reply
Use your mouse or fingers to pan + zoom. To edit the map, submit your ballot on November 6th.
alt_text_bot
12 days ago
reply
Use your mouse or fingers to pan + zoom. To edit the map, submit your ballot on November 6th.

Compile-time Dependency Injection With Go Cloud's Wire

1 Comment and 2 Shares

Overview

The Go team recently announced the open source project Go Cloud, with portable Cloud APIs and tools for open cloud development. This post goes into more detail about Wire, a dependency injection tool provided with Go Cloud.

What problem does Wire solve?

Dependency injection is a standard technique for producing flexible and loosely coupled code, by explicitly providing components with all of the dependencies they need to work. In Go, this often takes the form of passing dependencies to constructors:

// NewUserStore returns a UserStore that uses cfg and db as dependencies.
func NewUserStore(cfg *Config, db *mysql.DB) (*UserStore, error) {...}

This technique works great at small scale, but larger applications can have a complex graph of dependencies, resulting in a big block of initialization code that's order-dependent but otherwise not very interesting. It's often hard to break up this code cleanly, especially because some dependencies are used multiple times. Replacing one implementation of a service with another can be painful because it involves modifying the dependency graph by adding a whole new set of dependencies (and their dependencies...), and removing unused old ones. In practice, making changes to initialization code in applications with large dependency graphs is tedious and slow.

Dependency injection tools like Wire aim to simplify the management of initialization code. You describe your services and their dependencies, either as code or as configuration, then Wire processes the resulting graph to figure out ordering and how to pass each service what it needs. Make changes to an application's dependencies by changing a function signature or adding or removing an initializer, and then let Wire do the tedious work of generating initialization code for the entire dependency graph.

Why is this part of Go Cloud?

Go Cloud's goal is to make it easier to write portable Cloud applications by providing idiomatic Go APIs for useful Cloud services. For example, blob.Bucket provides a storage API with implementations for Amazon's S3 and Google Cloud Storage (GCS); applications written using blob.Bucket can swap implementations without changing their application logic. However, the initialization code is inherently provider-specific, and each provider has a different set of dependencies.

For example, constructing a GCS blob.Bucket requires a gcp.HTTPClient, which eventually requires google.Credentials, while constructing one for S3 requires an aws.Config, which eventually requires AWS credentials. Thus, updating an application to use a different blob.Bucket implementation involves exactly the kind of tedious update to the dependency graph that we described above. The driving use case for Wire is to make it easy to swap implementations of Go Cloud portable APIs, but it's also a general-purpose tool for dependency injection.

Hasn't this been done already?

There are a number of dependency injection frameworks out there. For Go, Uber's dig and Facebook's inject both use reflection to do runtime dependency injection. Wire was primarily inspired by Java's Dagger 2, and uses code generation rather than reflection or service locators.

We think this approach has several advantages:

  • Runtime dependency injection can be hard to follow and debug when the dependency graph gets complex. Using code generation means that the initialization code that's executed at runtime is regular, idiomatic Go code that's easy to understand and debug. Nothing is obfuscated by an intervening framework doing "magic". In particular, problems like forgetting a dependency become compile-time errors, not run-time errors.
  • Unlike service locators, there's no need to make up arbitrary names or keys to register services. Wire uses Go types to connect components with their dependencies.
  • It's easier to avoid dependency bloat. Wire's generated code will only import the dependencies you need, so your binary won't have unused imports. Runtime dependency injectors can't identify unused dependencies until runtime.
  • Wire's dependency graph is knowable statically, which provides opportunities for tooling and visualization.

How does it work?

Wire has two basic concepts: providers and injectors.

Providers are ordinary Go functions that "provide" values given their dependencies, which are described simply as parameters to the function. Here's some sample code that defines three providers:

// NewUserStore is the same function we saw above; it is a provider for UserStore,
// with dependencies on *Config and *mysql.DB.
func NewUserStore(cfg *Config, db *mysql.DB) (*UserStore, error) {...}

// NewDefaultConfig is a provider for *Config, with no dependencies.
func NewDefaultConfig() *Config {...}

// NewDB is a provider for *mysql.DB based on some connection info.
func NewDB(info *ConnectionInfo) (*mysql.DB, error) {...}

Providers that are commonly used together can be grouped into ProviderSets. For example, it's common to use a default *Config when creating a *UserStore, so we can group NewUserStore and NewDefaultConfig in a ProviderSet:

var UserStoreSet = wire.ProviderSet(NewUserStore, NewDefaultConfig)

Injectors are generated functions that call providers in dependency order. You write the injector's signature, including any needed inputs as arguments, and insert a call to wire.Build with the list of providers or provider sets that are needed to construct the end result:

func initUserStore() (*UserStore, error) {
    // We're going to get an error, because NewDB requires a *ConnectionInfo
    // and we didn't provide one.
    wire.Build(UserStoreSet, NewDB)
    return nil, nil  // These return values are ignored.
}

Now we run go generate to execute wire:

$ go generate
wire.go:2:10: inject initUserStore: no provider found for ConnectionInfo (required by provider of *mysql.DB)
wire: generate failed

Oops! We didn't include a ConnectionInfo or tell Wire how to build one. Wire helpfully tells us the line number and types involved. We can either add a provider for it to wire.Build, or add it as an argument:

func initUserStore(info ConnectionInfo) (*UserStore, error) {
    wire.Build(UserStoreSet, NewDB)
    return nil, nil  // These return values are ignored.
}

Now `go generate` will create a new file with the generated code:

// File: wire_gen.go
// Code generated by Wire. DO NOT EDIT.
//go:generate wire
//+build !wireinject

func initUserStore(info ConnectionInfo) (*UserStore, error) {
    defaultConfig := NewDefaultConfig()
    db, err := NewDB(info)
    if err != nil {
        return nil, err
    }
    userStore, err := NewUserStore(defaultConfig, db)
    if err != nil {
        return nil, err
    }
    return userStore, nil
}

Any non-injector declarations are copied into the generated file. There is no dependency on Wire at runtime: all of the written code is just normal Go code.

As you can see, the output is very close to what a developer would write themselves. This was a trivial example with just three components, so writing the initializer by hand wouldn't be too painful, but Wire saves a lot of manual toil for components and applications with more complex dependency graphs.

How can I get involved and learn more?

The Wire README goes into more detail about how to use Wire and its more advanced features. There's also a tutorial that walks through using Wire in a simple application.

We appreciate any input you have about your experience with Wire! Go Cloud's development is conducted on GitHub, so you can file an issue to tell us what could be better. For updates and discussion about the project, join the project’s mailing list.

Thank you for taking the time to learn about Go Cloud's Wire. We’re excited to work with you to make Go the language of choice for developers building portable cloud applications.

Read the whole story
kbrint
14 days ago
reply
Neat!
Share this story
Delete

In a short number of years this story will sound like madness

1 Comment

Canada installs Chinese underwater monitoring devices next to US nuclear submarine base

  • Ocean Network Canada confirms addition of hi-tech sensors built by Chinese scientists to its marine observatories in Pacific Ocean
  • US state department has ‘nothing to say’ on matter

Full story here, here is some further context from the piece:

Whatever the devices end up being used for, Chen Hongqiao, a researcher at the Centre for Canadian Studies at Guangdong University of Foreign Studies in Guangzhou, said there was no doubting the sensitivity of the issue.

“Deep sea observation networks are highly sensitive, and closely related to national security,” he said. “Countries don’t open them up to third parties unless there is a high level of trust and confidence.”

The decision to give China such access could have only come from highest corridors of power on both sides, he said.

“Such collaboration is very unusual. The implications go far beyond science, [so] it could have only happened with a nod from the top on both sides.”

The post In a short number of years this story will sound like madness appeared first on Marginal REVOLUTION.

Read the whole story
kbrint
16 days ago
reply
Curious...
Share this story
Delete

China's Hacking of the Border Gateway Protocol

1 Share

This is a long -- and somewhat technical -- paper by Chris C. Demchak and Yuval Shavitt about China's repeated hacking of the Internet Border Gateway Protocol (BGP): "China's Maxim ­ Leave No Access Point Unexploited: The Hidden Story of China Telecom's BGP Hijacking."

BGP hacking is how large intelligence agencies manipulate Internet routing to make certain traffic easier to intercept. The NSA calls it "network shaping" or "traffic shaping." Here's a document from the Snowden archives outlining how the technique works with Yemen.

Read the whole story
kbrint
16 days ago
reply
Share this story
Delete

Two Kinds of Technology Change

1 Share
Published on Thu Oct 11 2018 04:54:50 GMT+0000 (UTC)

Many of the examples in this post are drawn from Volume I of Fernand Braudel’s “Civilization and Capitalism: 15th-18th Century”, which I strongly recommend for anyone interested in a quantitative approach to history.

You may have heard that Gutenberg revolutionized the intellectual world with his invention of moving type in the mid-1400’s.

Here’s the thing, though: Gutenberg was not the first to try movable type. The Chinese were using basic printing presses in the ninth century; Pi Cheng introduced movable characters between 1040 and 1050. So why didn’t it catch on then? And even setting that aside, surely some tired monk must have thought of the idea sooner.

Turns out, prior to the 14th century, books were primarily printed on parchment — created from sheep skins. A single 150-page book required the skins of 12 sheep to make the parchment. That much parchment wasn’t cheap — the parchment on which a book was written cost far more than the actual writing. With that much cost sunk in the materials, it’s no wonder that book-buyers wanted beautiful, handwritten script — it added relatively little to the cost.

It was paper which changed all that. European paper production didn’t get properly underway until the 1300’s. Once it did, book prices plummeted, writing became the primary expense of book production, and printing presses with movable type followed a century later.

The printing press offers a clear example of a technology change whose arrival was limited, not by the genius of the inventor, but by economic viability. The limiting factor wasn’t insight, it was prices.

Once you go looking for it, there’s a lot of technology shifts like this. Newcomen’s steam engine (and Heron’s, long before). Schwenteer’s telegraph. Bushnell’s submarine. Babbage and Lovelace had all the key ideas for the modern computer in the 1820’s, but it wasn’t until the 1890 census that somebody wanted to pay for such a thing. And of course, Moore’s Law led to all sorts of ideas going from unprofitable to ubiquitous in the span of a decade or two.

In all these cases, the pattern is the same: the idea for an invention long predates the price shifts which make it marketable.

On the other hand, this isn’t the case for all technological progress. There are some technologies for which demand preceded capability. After some insight or breakthrough made the technology possible, adoption followed rapidly. Consider the Wright brothers’ flyer, or Edison’s lightbulb. Both had badly inferior predecessors, which didn’t really solve the problem: gliders and hot-air balloons for the Wright brothers, arc lights for Edison. Both built fast iteration platforms, tested a large possibility space, and eventually found a design which worked. And both saw rapid adoption once the invention was made.

One notable feature of these breakthrough-type technologies: the economic incentive for flight or the lightbulb was in place long before the invention, so of course many people tried to solve the problems. Both Edison and the Wright brothers were preceded by many others who tried and failed.

Here’s a simple model: technology determines the limits of what’s possible, the constraints on economic activity. We can think of these constraints as planes in some high-dimensional space of economic production. Economic incentives push us as far as we can go along some direction, until we run in to one of these constraints — and the technology we use depends on what constraint we hit.

Following the incentive gradient in the diagram above, we end up at the smiley face — using a mix of technologies A and B. This point is insensitive to small changes in the incentive gradient — the prices can shift a bit one way or the other, shifting the incentive gradient slightly, and the smiley-face point will still be optimal.

However, if prices shift enough, then we can see a sudden change.

Once the incentive gradient moves “down” sufficiently, we suddenly jump from the A-B intersection being optimal to the B-C intersection being optimal. A new set of constraints kicks in; we switch from technology A to technology C. That’s the printing press: inventing C doesn’t matter until the prices shift.

On the other hand, we can also change technologies by relaxing a constraint. Suppose some new-and-improved version of technology A comes along:

Technology A’ allows us to ignore the old A constraint, and move further along that direction. If we were using A before, then we’ll definitely want to switch to A’ right away. That’s Edison’s lightbulb.

In order for a technology to go from not-used to used, one of these two situations must hold: either the technology was unprofitable before and a price shift makes it profitable, or else it was profitable before, and many people tried to figure it out but couldn’t. If you yourself want to market some kind of technology, then you should consider which of these two situations applies. Has a recent price shift made it profitable? Have you made some sort of breakthrough which others have tried and failed to find? If the answer to both of those questions is no, then the technology will probably remain unused. If the answer to at least one of those questions is yes, then you may be on to something.



Discuss
Read the whole story
kbrint
30 days ago
reply
Share this story
Delete
Next Page of Stories