<p>Hi, I get this error after a few seconds. I just need that my worker can add new jobs to pool, is it posible?</p>
<pre><code>package main
import (
"flag"
"fmt"
"os"
"sync"
"time"
)
type Link struct {
URI string
Level int
}
var wg sync.WaitGroup
func main() {
uri := flag.String("uri", "http://test.com", "URI to parse")
workers := flag.Int("w", 2, "Size workers pool")
flag.Parse()
links := make(chan *Link, 50)
filteredLinks := make(chan *Link, 50)
links <- &Link{URI: *uri}
go filterLinks(links, filteredLinks)
for i := 0; i < *workers; i++ {
go func() {
wg.Add(1)
for item := range links {
fmt.Printf("%s\n", item)
if item.Level > 2 {
break
}
time.Sleep(time.Second)
filteredLinks <- &Link{URI: item.URI, Level: item.Level + 1}
}
wg.Done()
}()
}
os.Stdout.Sync()
wg.Wait()
}
func filterLinks(out chan *Link, in chan *Link) {
wg.Add(1)
seen := make(map[string]bool)
for item := range in {
if !seen[item.URI] {
out <- item
}
seen[item.URI] = true
}
wg.Done()
}
</code></pre>
<p>This is what I get after compile & run</p>
<pre><code>&{http://test.com %!s(int=0)}
&{http://test.com %!s(int=1)}
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [semacquire]:
sync.runtime_Semacquire(0x1172d7c)
/usr/local/opt/go/libexec/src/runtime/sema.go:56 +0x39
sync.(*WaitGroup).Wait(0x1172d70)
/usr/local/opt/go/libexec/src/sync/waitgroup.go:131 +0x72
main.main()
/Users/thuglife/dev/parser.go:49 +0x1e3
goroutine 5 [chan receive]:
main.filterLinks(0xc4200560c0, 0xc420056120)
/Users/thuglife/dev/parser.go:56 +0x127
created by main.main
/Users/thuglife/dev/parser.go:27 +0x179
goroutine 6 [chan receive]:
main.main.func1(0xc4200560c0, 0xc420056120)
/Users/thuglife/dev/parser.go:33 +0x8b
created by main.main
/Users/thuglife/dev/parser.go:30 +0x1ae
goroutine 7 [chan receive]:
main.main.func1(0xc4200560c0, 0xc420056120)
/Users/thuglife/dev/parser.go:33 +0x8b
created by main.main
/Users/thuglife/dev/parser.go:30 +0x1ae
</code></pre>
<hr/>**评论：**<br/><br/>Justinsaccount: <pre><p>the deadlock is because filterLinks never finishes</p></pre>d05cfea: <pre><p>i get it. Also go func() in main never finishes too</p></pre>ImLopshire: <pre><p>The wait group in <code>filterLinks()</code> is superfluous. Ranging over a channel blocks until the channel is closed.</p></pre>ImLopshire: <pre><p>This seems like a very strange way to use a wait group. Look at this simple example of using worker pools: <a href="https://gobyexample.com/worker-pools" rel="nofollow">https://gobyexample.com/worker-pools</a></p></pre>tv64738: <pre><p>Those wg.Adds happen too late to be of any use.</p>
<p>The code is very confusing, but what you seem to be trying to do is a web crawler that fetches links, parses the pages, and adds discovered links as more work to do. You can't use a channel as queue for that, because you cannot guarantee the amount of outstanding work queued stays below your limit of 50.</p></pre>_crackling: <pre><p>/Users/<strong>thuglife</strong>/dev/parser.go:30 </p>
<p>haha, love it</p></pre>
