The Go Blog

Introducing the Developer Experience Working Group

10 April 2017

Over the last several years, Go's audience has shifted from early adopters to mainstream users. Today, our users come from a wide variety of backgrounds, experiences, and expectations. The needs of users are growing faster than the Go project can currently address them. To streamline the experience for first-time Go users, we've created the Developer eXperience Working Group (DXWG).

For the next three months, this group will work together on delivering:

• improvements to the Go installation experience
• better guidelines to help new users
• guides on tooling and developer environments (editors and IDEs)
• running user studies to systematically analyze and measure friction points
• improvements to the Go Tour and Go Playground

A secondary goal of the working group is to better understand how to involve the Go community in charting Go’s future. We hope that working groups – Go team members working alongside community members – will help Go scale its leadership and address user needs. We’ll learn from this experience and iterate.

The initial members of the working group are: Carmen Andoh, Chris Broadfoot, Francesc Campoy, Jaana Burcu Dogan, Steve Francia, Jess Frazelle, Bill Kennedy, Katrina Owen, Natalie Pistunovich, Mat Ryer, Dmitry Shuralyov.

We are looking for additional people to help with contributing code, writing documentation, sharing feedback and experiences (user stories), reviewing contributions, and more. If you are interested in any of our current areas of focus, please join us on the #devexp channel on the Gophers Slack and subscribe to the golang-devexp mailing list.

By The Developer Experience Working Group

HTTP/2 Server Push

24 March 2017

Introduction

HTTP/2 is designed to address many of the failings of HTTP/1.x. Modern web pages use many resources: HTML, stylesheets, scripts, images, and so on. In HTTP/1.x, each of these resources must be requested explicitly. This can be a slow process. The browser starts by fetching the HTML, then learns of more resources incrementally as it parses and evaluates the page. Since the server must wait for the browser to make each request, the network is often idle and underutilized.

To improve latency, HTTP/2 introduced server push, which allows the server to push resources to the browser before they are explicitly requested. A server often knows many of the additional resources a page will need and can start pushing those resources as it responds to the initial request. This allows the server to fully utilize an otherwise idle network and improve page load times.

At the protocol level, HTTP/2 server push is driven by PUSH_PROMISE frames. A PUSH_PROMISE describes a request that the server predicts the browser will make in the near future. As soon as the browser receives a PUSH_PROMISE, it knows that the server will deliver the resource. If the browser later discovers that it needs this resource, it will wait for the push to complete rather than sending a new request. This reduces the time the browser spends waiting on the network.

Server Push in net/http

Go 1.8 introduced support for pushing responses from an http.Server. This feature is available if the running server is an HTTP/2 server and the incoming connection uses HTTP/2. In any HTTP handler, you can assert if the http.ResponseWriter supports server push by checking if it implements the new http.Pusher interface.

For example, if the server knows that app.js will be required to render the page, the handler can initiate a push if http.Pusher is available:

    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        if pusher, ok := w.(http.Pusher); ok {
            // Push is supported.
            if err := pusher.Push("/app.js", nil); err != nil {
                log.Printf("Failed to push: %v", err)
            }
        }
        // ...
    })

The Push call creates a synthetic request for /app.js, synthesizes that request into a PUSH_PROMISE frame, then forwards the synthetic request to the server's request handler, which will generate the pushed response. The second argument to Push specifies additional headers to include in the PUSH_PROMISE. For example, if the response to /app.js varies on Accept-Encoding, then the PUSH_PROMISE should include an Accept-Encoding value:

    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        if pusher, ok := w.(http.Pusher); ok {
            // Push is supported.
            options := &http.PushOptions{
                Header: http.Header{
                    "Accept-Encoding": r.Header["Accept-Encoding"],
                },
            }
            if err := pusher.Push("/app.js", options); err != nil {
                log.Printf("Failed to push: %v", err)
            }
        }
        // ...
    })

A fully working example is available at:

$ go get golang.org/x/blog/content/h2push/server

If you run the server and load https://localhost:8080, your browser's developer tools should show that app.js and style.css were pushed by the server.

Start Your Pushes Before You Respond

It's a good idea to call the Push method before sending any bytes of the response. Otherwise it is possible to accidentally generate duplicate responses. For example, suppose you write part of an HTML response:

<html>
<head>
    <link rel="stylesheet" href="a.css">...

Then you call Push("a.css", nil). The browser may parse this fragment of HTML before it receives your PUSH_PROMISE, in which case the browser will send a request for a.css in addition to receiving your PUSH_PROMISE. Now the server will generate two responses for a.css. Calling Push before writing the response avoids this possibility entirely.

When To Use Server Push

Consider using server push any time your network link is idle. Just finished sending the HTML for your web app? Don't waste time waiting, start pushing the resources your client will need. Are you inlining resources into your HTML file to reduce latency? Instead of inlining, try pushing. Redirects are another good time to use push because there is almost always a wasted round trip while the client follows the redirect. There are many possible scenarios for using push -- we are only getting started.

We would be remiss if we did not mention a few caveats. First, you can only push resources your server is authoritative for -- this means you cannot push resources that are hosted on third-party servers or CDNs. Second, don't push resources unless you are confident they are actually needed by the client, otherwise your push wastes bandwidth. A corollary is to avoid pushing resources when it's likely that the client already has those resources cached. Third, the naive approach of pushing all resources on your page often makes performance worse. When in doubt, measure.

The following links make for good supplemental reading:

• HTTP/2 Push: The Details
• Innovating with HTTP/2 Server Push
• Cache-Aware Server Push in H2O
• The PRPL Pattern
• Rules of Thumb for HTTP/2 Push
• Server Push in the HTTP/2 spec

Conclusion

With Go 1.8, the standard library provides out-of-the-box support for HTTP/2 Server Push, giving you more flexibility to optimize your web applications.

Go to our HTTP/2 Server Push demo page to see it in action.

By Jaana Burcu Dogan, Tom Bergan

Go 2016 Survey Results

6 March 2017

Thank you

This post summarizes the result of our December 2016 user survey along with our commentary and insights. We are grateful to everyone who provided their feedback through the survey to help shape the future of Go.

Programming background

Of the 3,595 survey respondents, 89% said they program in Go at work or outside of work, with 39% using Go both at home and at work, 27% using Go only at home, and 23% using Go only at work.

We asked about the areas in which people work. 63% said they work in web development, but only 9% listed web development alone. In fact, 77% chose two or more areas, and 53% chose three or more.

We also asked about the kinds of programs people write in Go. 63% of respondents write command-line programs, 60% write API or RPC services, and 52% write web services. Like in the previous question, most made multiple choices, with 85% choosing two or more and 72% choosing three or more.

We asked about people’s expertise and preference among programming languages. Unsurprisingly, Go ranked highest among respondents’ first choices in both expertise (26%) and preference (62%). With Go excluded, the top five first choices for language expertise were Python (18%), Java (17%), JavaScript (13%), C (11%), and PHP (8%); and the top five first choices for language preference were Python (22%), JavaScript (10%), C (9%), Java (9%), and Ruby (7%). Go is clearly attracting many programmers from dynamic languages.

Reading the data: This question was “multiple choice,” so the percentages add up to well over 100%. All graphs in this post show both the total count and the corresponding percentage of the 3,595 surveys completed.

Reading the data: This question was “ordered choice.” The first, second, third, fourth, and fifth choices are displayed as progressively lighter sections of the bars. The total count shown next to the bar is for all choices; the percentage list shows how the choices are divided.

Go usage

Users are overwhelmingly happy with Go: they agree that they would recommend Go to others by a ratio of 19:1, that they’d prefer to use Go for their next project (14:1), and that Go is working well for their teams (18:1). Fewer users agree that Go is critical to their company’s success (2.5:1).

When asked what they like most about Go, users most commonly mentioned Go’s simplicity, ease of use, concurrency features, and performance. When asked what changes would most improve Go, users most commonly mentioned generics, package versioning, and dependency management. Other popular responses were GUIs, debugging, and error handling.

When asked about the biggest challenges to their own personal use of Go, users mentioned many of the technical changes suggested in the previous question. The most common themes in the non-technical challenges were convincing others to use Go and communicating the value of Go to others, including management. Another common theme was learning Go or helping others learn, including finding documentation like getting-started walkthroughs, tutorials, examples, and best practices.

Some representative common feedback, paraphrased for confidentiality:

“The documentation is not clear enough for beginners. It needs more examples and often assumes experience with other languages and various computer science topics.”

“I want to use Go at work but struggle to convince my team to even try Go.”

“I can’t get management approval to use Go; they don’t see its value and worry about adoption and finding developers.”

We appreciate the feedback given to identify these challenges faced by our users and community. In 2017 we are focusing on addressing these issues and hope to make as many significant improvements as we can. We welcome suggestions and contributions from the community in making these challenges into strengths for Go.

Reading the data: This question asked how strongly the respondent agreed or disagreed with the statement. The responses for each statement are displayed as sections of a single bar, from “strongly disagree” in deep red on the left end to “strongly agree” in deep blue on the right end. The bars use the same scale as the rest of the graphs, so they can (and do, especially later in the survey) vary in overall length due to lack of responses. The ratio after the text compares the number of respondents who agreed (including “somewhat agree” and “strongly agree”) to those who disagreed (including “somewhat disagree” and “strongly disagree”). For example, the ratio of respondents agreeing that they would recommend Go to respondents disagreeing was 19 to 1.

Reading the data: This question asked for write-in responses. The bars above show the fraction of surveys mentioning common words or phrases. Only words or phrases that appeared in twenty or more surveys are listed, and meaningless common words or phrases like “the” or “to be” are omitted. The displayed results do overlap: for example, the 287 responses that mentioned “standard library” do include the 27 listed separately that mentioned “great standard library.” However, nearly or completely redundant shorter entries are omitted: there are not twenty or more surveys that listed “standard” without mentioning “standard library,” so there is no separate entry for “standard.”

Development and deployment

When asked which operating systems they develop Go on, 63% of respondents say they use Linux, 44% use MacOS, and 19% use Windows, with multiple choices allowed and 49% of respondents developing on multiple systems. The 51% of responses choosing a single system split into 29% on Linux, 17% on MacOS, 5% on Windows, and 0.2% on other systems.

Go deployment is roughly evenly split between privately managed servers and hosted cloud servers.

Working Effectively

We asked how strongly people agreed or disagreed with various statements about Go. Users most agreed that Go’s performance meets their needs (57:1 ratio agree versus disagree), that they are able to quickly find answers to their questions (20:1), and that they are able to effectively use Go’s concurrency features (14:1). On the other hand, users least agreed that they are able to effectively debug uses of Go’s concurrency features (2.7:1).

Users mostly agreed that they were able to quickly find libraries they need (7.5:1). When asked what libraries are still missing, the most common request by far was a library for writing GUIs. Another popular topic was requests around data processing, analytics, and numerical and scientific computing.

Of the 30% of users who suggested ways to improve Go’s documentation, the most common suggestion by far was more examples.

The primary sources for Go news are the Go blog, Reddit’s /r/golang and Twitter; there may be some bias here since these are also how the survey was announced.

The primary sources for finding answers to Go questions are the Go web site, Stack Overflow, and reading source code directly.

The Go Project

55% of respondents expressed interest in contributing in some way to the Go community and projects. Unfortunately, relatively few agreed that they felt welcome to do so (3.3:1) and even fewer felt that the process was clear (1.3:1). In 2017, we intend to work on improving the contribution process and to continue to work to make all contributors feel welcome.

Respondents agree that they are confident in the leadership of the Go project (9:1), but they agree much less that the project leadership understands their needs (2.6:1), and they agree even less that they feel comfortable approaching project leadership with questions and feedback (2.2:1). In fact, these were the only questions in the survey for which more than half of respondents did not mark “somewhat agree”, “agree”, or “strongly agree” (many were neutral or did not answer).

We hope that the survey and this blog post convey to those of you who are aren’t comfortable reaching out that the Go project leadership is listening. Throughout 2017 we will be exploring new ways to engage with users to better understand their needs.

Community

At the end of the survey, we asked some demographic questions. The country distribution of responses roughly matches the country distribution of site visits to golang.org, but the responses under-represent some Asian countries. In particular, India, China, and Japan each accounted for about 5% of the site visits to golang.org in 2016 but only 3%, 2%, and 1% of survey responses.

An important part of a community is making everyone feel welcome, especially people from under-represented demographics. We asked an optional question about identification across a few diversity groups. 37% of respondents left the question blank and 12% of respondents chose “I prefer not to answer”, so we cannot make many broad conclusions from the data. However, one comparison stands out: the 9% who identified as underrepresented agreed with the statement “I feel welcome in the Go community” by a ratio of 7.5:1, compared with 15:1 in the survey as a whole. We aim to make the Go community even more welcoming. We support and are encouraged by the efforts of organizations like GoBridge and Women Who Go.

The final question on the survey was just for fun: what’s your favorite Go keyword? Perhaps unsurprisingly, the most popular response was go, followed by defer, func, interface, and select.

By Steve Francia, for the Go team

Go 1.8 is released

16 February 2017

Today the Go team is happy to announce the release of Go 1.8. You can get it from the download page. There are significant performance improvements and changes across the standard library.

The compiler back end introduced in Go 1.7 for 64-bit x86 is now used on all architectures, and those architectures should see significant performance improvements. For instance, the CPU time required by our benchmark programs was reduced by 20-30% on 32-bit ARM systems. There are also some modest performance improvements in this release for 64-bit x86 systems. The compiler and linker have been made faster. Compile times should be improved by about 15% over Go 1.7. There is still more work to be done in this area: expect faster compilation speeds in future releases.

Garbage collection pauses should be significantly shorter, usually under 100 microseconds and often as low as 10 microseconds.

The HTTP server adds support for HTTP/2 Push, allowing servers to preemptively send responses to a client. This is useful for minimizing network latency by eliminating roundtrips. The HTTP server also adds support for graceful shutdown, allowing servers to minimize downtime by shutting down only after serving all requests that are in flight.

Contexts (added to the standard library in Go 1.7) provide a cancelation and timeout mechanism. Go 1.8 adds support for contexts in more parts of the standard library, including the database/sql and net packages and Server.Shutdown in the net/http package.

It's now much simpler to sort slices using the newly added Slice function in the sort package. For example, to sort a slice of structs by their Name field:

sort.Slice(s, func(i, j int) bool { return s[i].Name < s[j].Name })

Go 1.8 includes many more additions, improvements, and fixes. Find the complete set of changes, and more information about the improvements listed above, in the Go 1.8 release notes.

To celebrate the release, Go User Groups around the world are holding release parties this week. Release parties have become a tradition in the Go community, so if you missed out this time, keep an eye out when 1.9 nears.

Thank you to over 200 contributors who helped with this release.

By Chris Broadfoot

Participate in the 2016 Go User Survey and Company Questionnaire

13 December 2016

The Go project wants to hear from you!

The Go project wants to hear from you! Our goal is to create the best language for developing simple, reliable, scalable software. We are asking you to help by participating in a survey and if applicable, a company questionnaire.

The Go User Survey

Who: If you use Go, have ever used Go, have ever stopped using Go, or have any interest in the language, please help by sharing your feedback to improve Go for you and your fellow Gophers.

Where: Please take this 20-minute survey by December 22nd: Go User Survey 2016

The survey is anonymous and confidential.

Why: The Go project leadership depends on your feedback to guide the future of the Go project. Your responses will help to understand how we are doing and help plan improvements for the Go language, libraries, and tools.

After the survey closes, we will publish the anonymous aggregate results to the Go blog.

The Go Company Questionnaire

Who: If you are in a position to share details like “company name”, “if your company is hiring Go developers”, and “reasons your team or company adopted Go” then please help us by taking this questionnaire. We only need one response per company (or department for larger companies).

Where: Please take this 5-minute questionnaire by December 22nd: Go Company Questionnaire 2016

The questionnaire is confidential, but not anonymous.

Why: The Go project would like to better understand the companies using Go.

After the questionnaire closes the Go project leadership will use this information to better understand how companies are utilizing Go and in what ways the project can improve their experience. If you participate we may reach out for further information.

Spread the word!

Please help us spread the word by sharing this post on your social network feeds, at meetups, around your office and in other communities.

The survey & questionnaire close by December 22nd so please share today.

By Steve Francia